Asfaar Humanoid Soccer Simulation Team Description Paper

Seyyed Alireza Mirzargar, Seyyed Soroush HosseinAlipour, sajjad torabian seyed mehran tavakolian

Abstract This paper describes ASFAAR-3D humanoid soccer simulation . We explain some facts about the simulation base, the basic skills that a humanoid robot should contain and some methods that we used for producing our skills.

Keywords: Humanoid Robot, Walking Pattern, Kinematic, RoboCup Soccer Simulation

Introduction

Starting from a point to describe 3D soccer simulation, it is notable that a soccer agent thinks and decides about which action (In the set of all possible actions: walking, turning, shooting, standing up, etc.) should be done next. To determine what should be done as the next action, the agent must pay attention to a plenty of parameters (including: agent's position, ball's position, gyro rate, etc) Also, in order to have a good play, there should be some kind of cooperation between the agents. First of all, we produced skills with two methods: Reinforcement Learning and trajectory method. Our walk skill and back-walk skill used trajectory method and other skills used Reinforcement Learning.

Walking

Foot Trajectories

Assuming that the period necessary for one walking step is Tc, the time of the kth step is from kTc to (k+1)Tc, k=1,2,...,K Where K is the number of steps.

To simplify our analysis, The kth walking step begins when the heel of the right foot leaves the ground at t = kTc and ends when the heel of the right foot first touches the ground at t = (k + 1)Tc. In the following, we discuss on only the generation of the right foot trajectory. The left foot trajectory is the same except for a Tc delay.

For a sagittal plane, each foot trajectory can be denoted by vector $[x_{\mathbf{a}}(t), z_{\mathbf{a}}(t), \theta_{\mathbf{a}}(t)]^{\mathrm{T}}$ where $(x_{\mathbf{a}}(t), z_{\mathbf{a}}(t))$ is the coordinate of the ankle position, and $\theta_{\mathbf{a}}(t)$ is the angle of the foot.

In making the foot trajectory following parameters are involved:

$$\theta_{\rm a}(t) = \begin{cases} q_{\rm gs}(k), & t = kT_{\rm c} \ q_{\rm b}, & t = kT_{\rm c} + T_{\rm d} \ -q_{\rm f}, & t = (k+1)T_{\rm c} \ -q_{\rm ge}(k), & t = (k+1)T_{\rm c} + T_{\rm d} \end{cases}$$

$$x_{\mathbf{a}}(t) = \begin{cases} kD_{\mathbf{s}}, & t = kT_{\mathbf{c}} \ kD_{\mathbf{s}} + l_{\mathbf{a}\mathbf{n}} \sin q_{\mathbf{b}} + l_{\mathbf{a}\mathbf{f}} (1 - \cos q_{\mathbf{b}}), & t = kT_{\mathbf{c}} + T_{\mathbf{d}} \ kD_{\mathbf{s}} + L_{\mathbf{a}\mathbf{o}}, & t = kT_{\mathbf{c}} + T_{\mathbf{m}} \ (k+2)D_{\mathbf{s}} - l_{\mathbf{a}\mathbf{n}} \sin q_{\mathbf{f}} - l_{\mathbf{a}\mathbf{b}} (1 - \cos q_{\mathbf{f}}), & t = (k+1)T_{\mathbf{c}} \ (k+2)D_{\mathbf{s}}, & t = (k+1)T_{\mathbf{c}} + T_{\mathbf{d}} \end{cases}$$

$$z_{\mathbf{a}}(t) = \begin{cases} h_{\mathrm{gs}}(k) + l_{\mathrm{an}}, & t = kT_{\mathrm{c}} \ h_{\mathrm{gs}}(k) + l_{\mathrm{af}} \sin q_{\mathrm{b}} + l_{\mathrm{an}} \cos q_{\mathrm{b}}, & t = kT_{\mathrm{c}} + T_{\mathrm{d}} \ H_{\mathrm{ao}}, & t = kT_{\mathrm{c}} + T_{\mathrm{m}} \ h_{\mathrm{ge}}(k) + l_{\mathrm{ab}} \sin q_{\mathrm{f}} + l_{\mathrm{an}} \cos q_{\mathrm{f}}, & t = (k+1)T_{\mathrm{c}} \ h_{\mathrm{ge}}(k) + l_{\mathrm{an}}, & t = (k+1)T_{\mathrm{c}} + T_{\mathrm{d}} \end{cases}$$

$$\begin{cases} \dot{\theta}{\rm a}(kT{\rm c}) = 0 \ \dot{\theta}{\rm a}((k+1)T{\rm c} + T_{\rm d}) = 0 \end{cases}$$

$$\begin{cases} \dot{x}{\rm a}(kT{\rm c}) = 0 \ \dot{x}{\rm a}((k+1)T{\rm c} + T_{\rm d}) = 0 \end{cases}$$

$$\begin{cases} \dot{z}{\rm a}(kT{\rm c}) = 0 \ \dot{z}{\rm a}((k+1)T{\rm c} + T_{\rm d}) = 0. \end{cases}$$

Hip Trajectory

The hip trajectory can be denoted by a vector $[x_{\mathbf{h}}(t), z_{\mathbf{h}}(t), \theta_{\mathbf{h}}(t)]^{\mathrm{T}}$ , where $(x_{\mathbf{h}}(t), z_{\mathbf{h}}(t))$ is the coordinate of the hip position and $\theta_{\mathbf{h}}(t)$ denotes the angle of the hip In making the hip trajectory following parameters are involved:

$$z_{\rm h}(t) = \begin{cases} H_{\rm h,min}, & t = kT_{\rm c} + 0.5T_{\rm d} \ H_{\rm h,max}, & t = kT_{\rm c} + 0.5(T_{\rm c} - T_{\rm d}) \ H_{\rm h,min}, & t = (k+1)T_{\rm c} + 0.5T_{\rm d}. \end{cases}$$

$$x_{\rm h}(t) = \begin{cases} kD_{\rm s} + x_{\rm ed}, & t = kT_{\rm c} \ (k+1)D_{\rm s} - x_{\rm sd}, & t = kT_{\rm c} + T_{\rm d} \ (k+1)D_{\rm s} + x_{\rm ed}, & t = (k+1)T_{\rm c}. \end{cases}$$

$$\begin{cases} 0.0 < x_{\rm sd} < 0.5 D_{\rm s} \ 0.0 < x_{\rm ed} 0.5 D_{\rm s}. \end{cases}$$

$$\begin{cases} \dot{x}{\rm h}(kT{\rm c}) = \dot{x}{\rm h}(kT{\rm c} + T_{\rm c}) \ \ddot{x}{\rm h}(kT{\rm c}) = \ddot{x}{\rm h}(kT{\rm c} + T_{\rm c}). \end{cases}$$

Behaviors

Soccer simulation server(rcssserver3d) sends some data to the agent. The agent should analyze the data and make decision to perform the best action. Then the agent sends the specified data back to the server.

The agents must contain some procedures which manage the their behaviors. Obviously, a normal human soccer player has got these abilities. For our purpose, the agent must have some basic skills.

We have provided our agents with two kinds of BasicSkills and AdvancedSkills. Each AdvancedSkill uses some BasicSkills, and each BasicSkill sets the properties of the joints directly and then the agent can move. The agent can be a GoalKeeper, Defender, MidFielder and Striker. In each situation, we have a plan. GoalKeeper plan, Defender plan, MidFielder plan and Striker Plan. It runs a plan according to the player type, and the plan chooses which AdvancedSkill and where it should be used for.

Humanoid BasicSkills

BasicSkills are some basic movements. Our program has some extra movements. As we said, we used Reinforcement Learning for producing these skills. Reinforcement learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to maximize some notion of long-term reward.