ITANDROIDS-3D Research Proposal

Jackson P. Matsuura, Thomás Cavicchioli Dias

Itandroids Research Group Instituto Tecnológico de Aeronáutica (ITA); Praça Marechal Eduardo Gomes, 50, Vila das Acácias, 12228-900 São José dos Campos, SP, Brazil

Abstract This research proposal presents the plans of research of the ITANDROIDS team for the RoboCup Soccer Simulation 3D. The research planned by ITANDROIDS includes the learning of humanoid basic movements, the comparison among RoboCup Soccer Simulation 3D Server and USARSIM and the learning of cooperative behavior for humanoid agents.

Introduction

The ITANDROIDS research group started to work with RoboCup Soccer Simulation 2D in august 2005 and with RoboCup Soccer Simulation 3D in November 2005. Until July 2006 the main objective of the ITANDROIDS research group was to understand the RoboCup categories and mainly the RoboCup Simulation Servers. So, the ITANDROIDS work in RoboCup was restricted to just to implement a team capable of playing a Soccer Game and most of the development of the ITANDROIDS in 2006 was just based on heuristics and empirical tests.

From July 2006 to the present date the ITANDROIDS research group started to prepare itself to really research in RoboCup and not just develop RoboCup teams. Simulation teams adopted Software Engineering standards in their codes and Control Theory, used in real robotics devices, are now being implemented in our algorithms. Also, some machine learning techniques are being tried to improve the performance of the ITANDROIDS agents.

Another great step to turn the ITANDROIDS effort from just development to research was the acquisition of two Humanoid Robots, one RoboNova-I and one Bioloid Comprehensive Kit. The RoboNova-I was already augmented with one CMU-CAM2 camera and ITANDROIDS is also qualified for Atlanta2007 in the RoboCup Humanoid Soccer League.

Research Branches

We have three main research fields for the new RoboCup Soccer Simulation 3D server. The first is the application of learning algorithms for basic skills, like walk, run, turn, shift walk and kick. The second is the comparison among the Soccer Simulation 3D server and the USARSIM with humanoid robots models. And the third is the learning of cooperative behavior with few agents.

Basic Movements Learning

In the RoboCup Soccer Simulation 2D and in the Soccer Simulation 3D up to version 0.5.2 the agents actions were just high level actions like turn, kick, catch and accelerate. The team members did not worry about how to perform these low level actions and should concentrate their efforts in the decision making and in the cooperative behavior of the agents, but with the introduction of the humanoid robot model on the Soccer Simulation 3D version 0.5.3 just performing a kick or a turn should be a challenge.

These low level skills or basic movements like forward walking, kicking and turning can be empirically adjusted; this was the ITANDROIDS approach for defining the basic movements of a RoboNova-I for the Humanoid Soccer League (Figure 1) and for defining the forward walking movement of the SoccerBot for the Soccer Simulation 3D stage 1 qualifying (Figure 2); or they can be learned from autonomous exploration of the possible movements of the robot.

One of the drawbacks of the learning methods, like the well know and wide used Reinforcement Learning, is the number of interactions and consequently the time needed to learn something useful. This time requirement makes the learning methods of limited used, mainly with real robots, but as the simulations can be accelerated and run in multiple machines the learning methods can be applied in simulated humanoid models and the results and the methodology can be later used in real robots.

One of the researches which ITANDRODS want to do with the new Soccer Simulation 3D server is just the application of learning methods for basic movements like forward and backward walking, turning, shifting left and right and kicking the ball. The learned movements should also be adapted to be used with the ITANDROIDS real robots.

Simulation Comparison

The second line of research is the comparison among the USARSIM, used in the RoboCup Rescue Simulation Virtual Robots, and the Soccer Simulation 3D with humanoid Robots. ITANDROIDS team recently modeled the RoboNova-I in the USARSIM to use the simulation for learning basic robot skills which would be implemented in the real RoboNova-I robot. The RoboNova-I in the USARSIM is show in figure 3.

SoccerSimulation3D and USARSIM will be compared using various criteria like the effort needed to add a new Robot Model, documentation, reproduction of physics phenomena, simulation speed, among others. These results should help ITANDROIDS and other researches which have real humanoid robots to choose the best simulation tool for help in their researches.

As ITANDROIDS already modeled a RoboNova-I for USARSIM we will also try to model a RoboNova-I for the Soccer Simulation 3D server and the SoccerBot to the USARSIM to start the comparisons. After some learning is done the movements obtained with Soccer Simulation 3D and USARSIM should be implemented in a real RoboNova-I as a final test.

Multi Agent Coordination

The soccer games with 11 agents in each team are very complex with too many possible states and the use of learning methods for the cooperative behavior of the agents should take too much time to achieve good results. In a simulation with fewer agents, like 2 to 5 in each team the search space is smaller and learning algorithms for cooperative behavior should present good results in less time than in 11 versus 11 simulations.

ITANDROIDS will start to use learning techniques for cooperative behavior with only two robots in each team and use up to five robots in each team, comparing the performance of the teams with the learned cooperative behavior with playing against a team implemented using heuristic and empirical cooperative decisions. With just few robots it is expected that the learning techniques present best results than with more agents.

Conclusion

The new version of the RoboCup Soccer Simulation 3D, with humanoid robots allow the research of basic movements, decision making and cooperative behavior of a team of humanoid robots, like in the RoboCup Humanoid Soccer League. This way the new server is not only a research tool by itself, but may also allow Humanoid teams to test their solutions and learn using the simulated humanoids.

As ITANDROIDS just created and qualified its team in the RoboCup Humanoid Soccer League the new Soccer Simulation Server 3D can be used to improve the team in the Humanoid Soccer League and also the research done with the real humanoids robots can be used to improve the research with the Soccer Simulation 3D. Just as an example, the control algorithm and the forward step movements implemented for the Soccer Simulation 3D Stage 1 qualifying were inspired in the work done with the RoboNova-I for the Humanoid League qualifying.

As ITANDROIDS works now in the Humanoid, in the Rescue Simulation Virtual Robots (already qualified for Atlanta2007) and in Soccer Simulation 3D and as the three leagues have many shared points we really think that we should really improve the research in these three leagues together.