The NUbots Team Description Paper 2019

Matthew Amos, Alex Biddulph, Stephan Chalup, Daniel Ginn, Alexandre Mendes, Josephus Paye, Ysobel Sims, Peter Turner, Taylor Young

Newcastle Robotics Laboratory School of Electrical Engineering and Computing Faculty of Engineering and Built Environment The University of Newcastle, Callaghan 2308, Australia

http://robots.newcastle.edu.au

Abstract The NUbots are the RoboCup team of the University of Newcastle, Australia. In 2019 they continue their participation in the Teen-Size Humanoid League using their self-printed robots based on the Igus-Nimbro design. In previous years, the NUbots participated in the Standard Platform League (2002-2011) and the Kid-Size Humanoid League (2012-2017). This year, the NUbots have participated in the Teen-Size Humanoid League, achieving third place. The NUbots' research addresses applications of machine learning, software engineering and computer vision. Their current research focus is Deep Learning. This paper provides an overview of the NUbots team, its software system and robot platform.

1 Introduction

For 16 years the Newcastle Robotics Laboratory has been hosting the NUbots robot soccer team as its central project. The NUbots previously competed at RoboCup in the Four-Legged League, the Standard Platform League and, until 2017, in the Kid-Size Humanoid League. In 2018, the NUbots debuted in the Teen-Size Humanoid League where they placed third. In 2019, the NUbots will be continuing their involvement in the Teen-Size Humanoid League.

The goal of the 2019 NUbots team is to improve upon their 2018 performance with further modifications to their NUgus robot design and the software. The NUgus robot is based on the Igus-Nimbro design developed by the University of Bonn [1].

The following sections describe research and history of the NUbots team. Some of the material, such as the history and previous research, was already covered by [8].

2 Scientific Aspects of the NUbots team

The scientific aspects of the current NUbots system involve the implementation of deep learning on robots, software engineering for robotics and AI, and various aspects of computer vision, localisation, human-robot interaction and machine learning.

2.1 Current Research Projects in the Newcastle Robotics Lab

Some ome of the NUbots' current projects are listed below. Additional details on the NUbots research in the past 16 years can be found in [8] and on the NUbots homepage.

Stereo Visual Mesh: Recent work in the lab has resulted in the development of a high framerate obstacle detector, named The Visual Mesh [4]. The Visual Mesh, used in conjunction with deep learning, is capable of detecting a soccer ball in a 1280 × 1024 image at framerates of up to 400 fps on a humanoid robot, with little loss of accuracy when the ball is at distance. Current research being developed by a PhD student aims at increasing the detection capabilities of the Visual Mesh by considering multiple objects, distance, and stereo vision.

Localisation Techniques for Robotics: Another PhD research project is using fast Deep Learning algorithms to semantically label important navigational features to enhance visual SLAM implementations like ORB-SLAM [10]. Preliminary tests of ORB-SLAM have demonstrated that it is capable of running in real-time on our CPU while the NUgus is walking on the field, and is able to maintain tracking under the swaying and jarring motions of humanoid locomotion. The results of this testing will be presented at the AI2018 conference in New Zealand¹ . We believe that ORB-SLAM will be able to provide localisation where visual odometry methods tested by NimbRo [3] proved unsuitable.

Multi-Objective Evolutionary Algorithms for Walk Optimisation: The NUbots walk engine and kick scripts were optimised by a Software Engineering Honours student using a Multi-Objective Evolutionary Algorithm (MOEA). The Gazebo simulation engine [7] was used to allow rapid optimisation without causing undue wear and tear on the physical robots. A generic MOEA framework was implemented and utilised the Ignition Transport library² to communicate with the Gazebo simulation engine.

When tested on the robot, the resulting script doubled the kick distance, without the robot falling over afterwards. Moreover, the robot's torso sway during the kick process was decreased by 50%. The MOEA framework was also tested on the walk engine, increasing the walk speed by 30%, and its stability by 40%. Further work on this system will improve the accuracy of the simulated servos as well as implementing more advanced learning techniques.

Semi-Synthetic Data Generation: A Physics-Based Rendering (PBR) pipeline has been developed to generate semi-synthetic data for deep learning purposes as part of a PhD project. The PBR generation tool utilises 360° High Dynamic Range (HDR) images to create realistic lighting for the constructed scene. Soccer balls, goals and the soccer field are synthetically generated, allowing the random placement of camera positions within the scene to create a large quantity of fully annotated semi-synthetic data.

The pipeline is capable of exporting either mono or stereo raw images, segmentation images, and depth images. Currently, the NUbots team has over 1 million stereo images with corresponding segmentation and depth ground truth images. An example output from the pipeline can be seen in Figure 1.

The PBR tool facilitates easy adaptation to custom scenes. Particularly, this has encouraged investigation of robot performance in outdoor and naturally lit environments. The pipeline is released completely open-source and can be found online [2].

2.2 Background and Research Interests of the NUbots Team Members

Matthew Amos is a fifth year undergraduate student studying a combined degree in Computer Science and Computer Engineering with Honours. He is interested in computer vision, machine learning, and synthetic data generation.

Alex Biddulph is studying for a Doctorate of Philosophy in Computer Engineering. Alex has undergraduate degrees in Computer Engineering and Computer Science with Honours in Computer Engineering. The focus of Alex's studies revolves around the symbiotic relationship between hardware and algorithms, with a focus on high framerate stereo vision for humanoid robots.

Associate Professor Stephan Chalup is the Head of the Newcastle Robotics Lab. His current research focuses on deep learning, neural information processing systems and topological data analysis.

Daniel Ginn is pursuing a PhD in Computer Science with a focus on localisation and mapping using robotic platforms in the context of RoboCup. He joined the NUbots competition team in 2017.

Dr. Alexandre Mendes is the Deputy Head of the Newcastle Robotics Lab. He is a Senior Lecturer in Computing and Information Technology. He joined the group in September 2011 and his research areas are evolutionary algorithms and optimisation.

Josephus Paye is a third year undergraduate student studying Computer Science, with a major in Computer Systems and Robotics. He contributes to NUsight, the web-based debugging environment.

Ysobel Sims is an undergraduate student studying a combined degree in Computer Science and Mathematics. She is interested in the mathematics behind humanoid robots, including balance and motion.

Peter Turner is technical staff in the School of Electrical Engineering and Computing. Peter provides hardware support and assists the team with physical robot design upgrades.

Taylor Young is a third year undergraduate student studying Electrical Engineering. He contributes to the hardware-based aspects, the design and manufacturing of the robots.

The current NUbots team acknowledges the input of team members of previous years and other colleagues from the Newcastle Robotics Lab. Details are linked to www.robots.newcastle.edu.au.

2.3 Related Research Concentrations

The Interdisciplinary Machine Learning Research Group (IMLRG) investigates different aspects of machine learning and data mining in theory, experiments and applications. The IMLRG's research areas include: Dimensionality reduction, vision processing, robotics control and learning, evolutionary computation, optimisation, reinforcement learning, kernel methods, and deep learning.

3 History of the NUbots and Prior Performance in RoboCup Competitions [8]

The NUbots team competed in the Four-Legged-League from 2002-2007 using Sony AIBO ERS-210 and ERS-7 robots. The NUbots participated for the first time at RoboCup 2002 in Fukuoka in the Sony Four-Legged League (third place). At RoboCup 2006 in Bremen, Germany, the NUbots won the title.

From 2008 to 2011 they used the Aldebaran Nao within the Standard Platform League. They achieved a first place in 2008 as part of the NUManoid team in Suzhou, China.

The NUbots joined the Kid-Size Humanoid League in 2012 with the DARwIn-OP robots, and ported their SPL codebase to the new platform. They retained their robust and fast vision and localisation systems from the SPL, and ported the B-human NAO walk to the DARwIn-OP for 2012-2013. From 2014-2016 the NUbots redeveloped their software system based on the NUClear software architecture [5]. For the DARwIn-OPs, they made small modifications of the head, feet and cameras. In 2017 the NUbots reached the quarter finals of the KId-Size Humanoid League using their mixed team of DARwIn-OP an Igus-Nimbro based robots [1]. After the competition, the team retired all DARwIn-OP robots, replacing them with their NUgus platform; a modified Igus-Nimbro design. In 2018, the NUbots competed in the Teen-Size Humanoid League for the first time using a team of their NUgus robots and finished in third place. In 2019, the NUbots will again compete in the Teen-Size Humanoid League with further improvements to their NUgus design.

4 Software and Hardware Overview

The NUbots team's software source is available from [9] and is covered under the GPL. This code includes associated toolkits for building and deploying the software. Our software is designed to work on multiple robotic platforms, and all of the individual modules have been designed to be easily used in other systems. The flexibility of our approach has been demonstrated in a deployment of the NUbots vision system on a maritime platform³ , and the adaptation of the codebase to support both the DARwIn-OP and Igus-Nimbro/NUgus platforms, encompassing both 32-bit and 64-bit CPU architectures with easy adaptability to ARM-based platforms.

Following development of a new software system in 2014-2017, the NUbots are now focusing on the RoboCup Teen-Size Humanoid League. Their research aims include efficient vision processing; improved localisation; generic ball detection; and improving the walk engine. The NUClear based NUbots software is designed to allow new teams and team members to easily understand and innovate on existing code, and is made freely available to encourage research and innovation. Recently, the ability to integrate modules implemented in Python has been incorporated into the NUbots codebase, further increasing the ease for new teams and team members to incorporate their code into the NUbots and NUClear systems.

5 Enhancements of the NUbots Hardware and Software Compared to the Previous Year

Based on comments from other teams at RoboCup 2018 and the NUbots own issues with the CM730 subcontroller, the NUbots have been developing a new subcontroller, as stated in Section 2.1.

The Gazebo simulation engine was integrated with the NUbots software architecture to facilitate simulation of the robots systems and to enable the use of genetic algorithms and reinforcement learning to find improved solutions to bipedal walking and other necessary humanoid motions.

5.1 Details on Software and Hardware used from Other Teams

Acknowledgement of the use of code for the walk engine: The NUbots robots used a walk engine based on the 2013 Team Darwin code release. We acknowledge the source of this code. The NUbots have ported this code to C++ and restructured the logic, making numerous structural and technical changes. The current code for the NUgus robot walk is a further modification and extension of the code that the NUbots used on their Darwins in previous years.

Acknowledgement of the use of robot design files: The NUbots NUgus robots are based on the 2016 Team Nimbro design files [6]. We acknowledge the source of this design. The NUbots have made modifications to the design to account for a different printing process and to improve the usability of the design. The modified design files are released under the original license [11].

The NUbots teams own contributions: The NUbots vision system now utilises the Visual Mesh algorithm [4]. This algorithm allows for fast, efficient, and accurate deep learning to be performed on high resolution images at frame rates in excess of 400 frames per second. The implemented algorithm provides guarantees over the number of sampled pixels within a given object and at any distance from the camera, allowing the user full control over the trade-off between the algorithms run time and accuracy.

Acknowledgement

The NUbots team is grateful to their main industrial partners and sponsors 4Tel Pty Ltd, Tribotix, Pritchard Electronics and the Morris Group. They also would like to thank the School of Electrical Engineering and Computing, and the Faculty of Engineering and Built Environment at The University of Newcastle, Australia, for their continuous support. Alex Biddulph is supported by an Australian Government Research Training Program scholarship and a top-up scholarship through 4Tel Pty Ltd.

References

1. Philipp Allgeuer, Hafez Farazi, Grzegorz Ficht, Michael Schreiber, and Sve Behnke. The igus humanoid open platform: A child-sized 3d printed open-source robot for research. German Journal on Artificial Intelligence (KI), 30:315–319, 2016.
2. M. Amos and A. Biddulph. Nubots pbr pipeline repository. https://github.com/nubots/NUpbr.
3. Hafez Farazi, Philipp Allgeuer, Grzegorz Ficht, Dmytro Pavlichenko, Diego Rodrguez, and Sven Behnke. The NimbRo team description paper 2018. In The official website of the RoboCup Humanoid League, 2018.
4. Trent Houliston and Stephan K. Chalup. Visual mesh: Real-time object detection using constant sample density. CoRR, abs/1807.08405, 2018.
5. Trent Houliston, Mitchell Metcalfe, and Stephan K. Chalup. A fast method for adapting lookup tables applied to changes in lighting colour. In RoboCup 2015: Robot World Cup XIX, volume 9513 of Lecture Notes in Artificial Intelligence (LNAI), pages 190–201. Springer, 2015.
6. igus GmbH. Humanoidopenplatform. https://github.com/igusGmbH/HumanoidOpenPlatform, January 2019.
7. N Koenig and A Howard. Design and use paradigms for Gazebo, an open-source multi-robot simulator. In Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on, volume 3, pages 2149–2154. IEEE.
8. Stephan Chalup Luke Farrawell Daniel Ginn Trent Houliston Alexandre Mendes Josephus Paye Peter Turner Matthew Amos, Alex Biddulph and Taylor Young. The NUbots team description paper 2018. In The official website of the RoboCup Humanoid League, 2018.
9. M. Metcalfe, J. Fountain, A. Sugo, T. Houliston, A. Biddulph, A. Dabson, T. Johnson, J. Johnson, B. Annable, S. Nicklin, S. Fenn, D. Budden, J. Walker, J. Reitveld, Y. Sims, T. Young, M. Amos, and D. Ginn. Nubots robocup code repository. https://github.com/nubots/NUbots.
10. Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5):1147–1163, 2015.
11. T. Young and A. Biddulph. Nubots hardware repository. https://github.com/nubots/NUware.