BraveCorpses 3D Soccer Team Description Paper

Hamed Shahbazi, Mohammad Masood Masaeli, Reza Ghanami, Reza Bashiri, Mehrdad Pedram, Meghdad Paknezhad

University of Sheykhbahaee, Isfahan, Iran

Abstract This paper describes the BraveCorpses team for the Robocup2011 competitions. In this paper the result of our efforts in intelligence and efficiency of robots in the simulation world will be explained. It contains running and optimizing of the motions such as walking and standing up by optimizing algorithms, and implementing a learning method named as CPG biped robots. Soon by improving the implementation of these techniques and adding better online decision making abilities you'll see the improvements of our robots' abilities.

1 Introduction

In the last few years, competitions called Robocop have been held which are founded to solve the problems and spread the knowledge of artificial intelligence and in order to absorb the cooperation of several groups, it has used the worlds' favorite sport, "soccer" and holds competitions in different levels.

We are also working in 3D soccer simulation branch to improve this knowledge and cooperate in making better ways of using machines in man's life. This team, named BraveCorpses is founded by some students and instructors from Sheykhbahaee university of Isfahan-Iran.

The problem of robot locomotion is where neuroscience and robotics converge. This common part is pattern generators in the spinal cord of vertebrate animals called "Central Pattern Generators" (CPGs). Central pattern generators are neural circuits located in the end parts of the brain and first parts of the spinal cord of a large number of animals and is responsible for generating rhythmic and periodic patterns of locomotion in different parts of the their bodies. Although these pattern generators use very simple sensory inputs imported from the sensory systems, they can produce high dimensional and complex patterns for walking, swimming, jumping, turning and other types of locomotion. The origin of many movements in animals is central pattern generators which were discovered by Brone in the early decades of the 20th century. He discovered that the movement in many animals is an outcome of central neuronal activities in some parts of their neural system, and simple sensory inputs change these activations and make them capable of responding to the extraneous perturbations [1]. The idea that CPGs are neural networks generating complex locomotion patterns with only simple inputs are a provocative one which is intended to model in this paper.

The other parts of this document contains robot model which is about the architecture and the way of matching the internal layers of the agent, then we have the discussion part that in brief represents some of discussions we made on different tasks and finally we'll see the conclusion and references.

2 Robot Model

We developed an architecture for implementing the robot to interact with Simpspark. This architecture contains three layers. These levels are Planning layer, High level layer and Low level layer and their relations showed in figure 1.

2.1 Planning layer

In this unit the agent makes decisions according to its place, situation and time. The agent gets the required information for these decisions from the Localizer, World Model, and Agent Model parts included in the library of bats and sends instructions to the high level skills layer, then this layer executes these instructions in the simulation environment by the means of the control layer which is a part of the library of bats as well.

2.1.1 DMU

In this unit the agent makes decisions according to its place, situation and time. The agent gets the required information for these decisions from the Localizer, World Model, and Agent Model parts included in the library of bats and sends instructions to the high level skills layer, then this layer executes these instructions in the simulation environment by the means of the control layer which is a part of the library of bats as well.

2.2 High Level Layer

This layer contains trajectory-based motion execution and learning-based motion generation capabilities.

2.2.1 Trajectories

In this technique, trajectories are already stored in a text file and the agent reads the values which in fact are called joints angels and sends related motion data to the control layer to execute. More explanations is in the following:

In the Simspark simulation environment the NAO robot has servo motors to which we can send angular velocity values to move. In this environment a time period is 20ms so each angular velocity value is the variety of a joint per 20 ms.

Trajectories are series of joint angles (in radians) stored in files (named motion files) and Every line of a motion file contains the situations of some joints in a time cycle in radians. Figure 2 is the format we are talking about:

Therefore we can convert these values to the angular velocity by the below relation.

Angular Velocity of a Joint = Stored Value in the Trajectory file - Current Angle

The most important problem is creating these files. An idea is to start with random values, but because of too many parameters, it's clear that the idea is to be failed! The other idea is to use the tools for making motions. One of the tools is Webots program that provides the ability to make any movement by the hand and stores it in the format that we talked about before. Webots is also a robot simulator software which supports the NAO robot which its simulation is close to the Simspark software.

2.2.2 Motion learning using Programmable CPG

In this section we will explain a motion learning method which we have used to train the curvilinear bipedal walk on the Simulated Nao Soccer Robot. We will explain what the curvilinear bipedal walk is and how we will use Generic CPGs to train this walk to our Nao robot. We will discuss the nonlinear dynamical oscillators and the fundamental prosperities of these oscillators. Then the coupling scheme and its roll in walking will be discussed.

The special type of locomotion we have focused on in this paper is curvilinear walking. The curvilinear walk is a smooth traveling from a point in the soccer field to a new point elongated a curve shape path. A special case of curve shape path is a part of circle. In Figure2 this kind of walk is shown. During a curvilinear walk the robot should synchronically rotate and travel forward and it must keep its distance from what it should be.

2.2.3 Procedures

Sometimes the movements are so simple and only happen in specific situations. For instance when the robot is born in the Simspark, in order to initialize, it should put its hands in a specific form. We can do this via procedures which send some specified data to the control layer.

2.3 Low Level Layer

In this layer we just use the bats library which models the world and omits the noises by the Kalman algorithm and its cerebelum module is really reliable to send things to the Simspark server.

3 Discussion

This section discusses the optimization and implementation of various motion techniques.

3.1 Optimizing Motions

As mentioned before, doing some movements are done via trajectories which is produced by Webots. But there are some differences between the modeling of NAO robot in Simspark and Webots and on the other hand as trajectories are made by hand, they have not a good ability to balance the robot and they include extra unusable movements. So we should make these trajectories optimized.

In this regard we started with Webots walking motion and modeled this skill, then optimized the model via Matlab tools.

3.1.1 Curve Fitting

According to the groups' discussions and research, the robots' joints movements in walking are periodic, so for a accurate modeling we used sum of some sinus functions for every joint as showed in the follow:

$a_1 \sin(b_1 x + c_1) + a_2 \sin(b_2 x + c_2) + a_3 \sin(b_3 x + c_3) + a_4 \sin(b_4 x + c_4) + a_5 \sin(b_5 x + c_5) + a_6 \sin(b_6 x + c_6) + a_7 \sin(b_7 x + c_7) + a_8 \sin(b_8 x + c_8)$

In these functions the "x" variable stands for the time," a" is the amplitude, "b" is the frequency and "c" is the initial phase. Figure 8 is the result of modeling for the left knee joint in walking.

3.1.2 Genetic Algorithm

After modeling a trajectory by curve fitting, it's time to optimize. So we used the Genetic Algorithm (GA) to do it. This algorithm has an input which is called the initial population and regarding to the fitness function, after a while it generates better populations and individuals which are useable for better movements.

To use this algorithm, we set the initial population from the result of curve fitting and defined a fitness function to communicate the parameters via TCP/IP to our agent so every new individual could be tested on the Simspark and the agent sends back the results to Matlab.

Figure 9 shows the relations between Simspark, Our Agent and Matlab:

4 Conclusion

We introduced a new method for generating curvilinear bipedal walking pattern using a Programmable Central Pattern Generator which is trained by NAO basic walking trajectories. This model can be used for other movements in Soccer matches between NAO robots in Robocup and because of this, it would be a very useful and standard method for these competitions. There is a large number of benefits in CPGs that can be applied in this method. This is what we are going to do in our future researches. We plan to implement our method in real NAO and test its efficiency.

References

1. A.J. Ijspeert, Central pattern generators for locomotion control in animals and robots: a review, Journal of Neural Networks,Elsivier, Vol 21/4, pp. 642-653, 2008.
2. J.J. Kuffner, S. Kagami, Dynamically-stable Motion Planning for Humanoid Robots, Journal of Autonomous Robots vol. 12, No. 1, pp. 105-118., 2002.
3. A.J. Beng Tay, Walking Nao Omnidirectional Bipedal Locomotion , Thesis in University of New South Wales, 2009.
4. A. Cherubini, F. Giannone, L. Iocchi, M. Lombardo, G. Oriolo, Policy gradient learning for a humanoid soccer robot, Journal of Robotics and Autonomous Systems 57, pp. 808-818, 2009.
5. M. Vukobratovic , D. Juricic, Contribution to the Synthesis of Biped Gait, IEEE Trans. On Bio-Medical Engineering, vol. BME-16, no. 1, pp.1-6,1969.
6. L. Righetti , A.J. Ijspeert, Programmable Central Pattern Generators:an application to biped locomotion control , Proceedings of the 2006 IEEE International Conference on Robotics and Automation Orlando, Florida - May 2006.
7. L. Righetti, J. Buchli, A.J. Ijspeert, Dynamic hebbian learning in adaptive frequency scillators, Journal of Physica D, pp. 105-116, 2005.
8. C. Lathion, Biped locomotion on the Hoap2 robot, Master's thesis, EPFL, 2006.
9. Joseph Giarratono. Gary Riley. Expert Systems Principles and Programming. Third Edition.