Nexus Robosoccer Development Process from 2D Simulated Agents through 3D Humanoid Soccerbots

Amin Milani Fard, Hossein Ansari, Vahid Salmani, Mahmoud Naghibzadeh, Hossein Kamyar

Software Simulation and Modeling Labaretory Ferdowsi Universioty, Mashhad, Iran

http://nexus.um.ac.ir/

Abstract Soccer simulation as an effort for motivating researchers in field of artificial intelligence and robotic research has always been a progressive approach. Robotic soccer is a particularly good domain for studying multi-agent systems and behaviors. In this paper, we describe researches done by Nexus team from the prior 2D simulation environment till curent humanoid simulation version. The main development features were done on decision making, action selection and coach strategy making modules using fuzzy logic mechanism and game theory approach. Some very basic humanoid actions are also explained.

Keywords: Soccer simulation, multi-agent systems, fuzzy logic systems, action selection mechanism, agent skill.

1 Introduction

Robotic soccer is a particularly good domain for studying multi-agent systems. It has been gaining popularity in recent years with international competitions like RoboCup which is planned for the near future [1]. Soccer simulation environment is a clientserver platform which provides an excellent testbed to develop multi-agent systems. With this testbed, researchers need not get involved with the complexities of physical robot developmets. In RoboCup simulation league, many teams of 11 autonomous software agents compete against each other by using RoboCup soccer server simulator software which is available from the official simulator website [2].

Nexus is the RoboCup Soccer Simulation of Ferdowsi University of Mashhad, Iran. Established in 2002, the team firstly participated in RoboCup contest in 2003 Padova, Italy in Soccer-2D league. Afterwards, NEXUS could go as high as the third round in RoboCup 2005 Osaka, Japan, and ranked 9th-12 th place among 33 teams. In this paper, we briefly proposed our research works done in the RoboCup simulation filed. Actually since humanoid simulation league is very new members just confined themeselves with simple exprimentral approaches in contrast with prior scientific approaches.

2 Our 2D environment approaches

One of the first leagues of RoboCup was the two-dimensional soccer simulation league. In fact, two-dimensional soccer simulation league helped to address many different open problems of creating cooperative multiagent systems. In such environment Nexus team focused on decision making and action selection module which is considered a high-level action. The best action is the one that helps towards the agent's utmost success. The attempt chosen has to bring about the most possible positive results in each simulation cycle, consistent with the definition of an ideal rational agent [3]. Every agent has to analyze various conditions as well as to handle newly received information. An intelligent agent should use the recently received information from the server in the best possible way. It is possible that parts of the received information from the surrounding be of no use or of little importance. Considering parameters of each of the three possible actions (shooting, dribbling, and passing), the information received from the surrounding area and the existing conditions can be divided into two parts: The information that is related to only one specific action and the information that is common among all three actions [4].

2.1 One-phase decision making mechanism

In our one-phase evaluation method, we use a specific weight for each parameter that affects an action. Through test runs and analysis of the outcomes, we have experimentally obtained proper weights for these parameters. The analysis was aimed at pinpointing the weaknesses of our team and trying to adjust the weights to improve the ability of the system. Each weight can be either a reward or a punishment whose summation for each one of the possible actions can result in a computed priority that recommends the most reasonable action. To obtain the weights, we start with an initial value for each weight. Afterward, the agent is made to contest several times and after each contest, the weights are readjusted. This process is similar to the supervised learning [3], but it is performed offline. The weights will gradually adjust to a stable value. To evaluate the priority for each one of the possible actions, both specific and common measures are used. The highest calculated priority determines the preferred action.

2.2 Two-phase decision making mechanism

To determine the best action from amongst all possible ones for a given situation, we first recognize the best of each action, i.e., the best shoot, the best dribble, and the best pass, independently. It is clear that, when the best possible shoot is sought the parameters that affect the shooting action are considered, only. For dribble and pass actions a similar process is followed. In the next phase, we select the best of bests, i.e., the system chooses the best action from amongst the three best actions shoot, dribble, and pass. In this phase, common measures are used in order to evaluate actions.

2.3 Fuzzy Two-phase decision making mechanism

We expected [5] the fuzzy system to be appropriate for decision-making process in the soccer simulation environment, considering the noise produced by the soccer server and uncertainties which affect all the perceptions and actions of the agents. Fuzzy systems are not sensitive to the completeness of the rule base, and even sometimes by removing half of the rules from a working system the performance does not degrade, as long as the boundary rules are preserved in the fuzzy associative memory [6]. Our fuzzy rule base includes 12 rules. The number of rules is much lower than the number of rules for our crisp system which was 50.

The proposed algorithm was implemented on Nexus soccer simulation team [4]. Results of ten games show that final scores of the team improved in the fuzzy approach. A team's success is directly influenced by each agent's actions. To calculate an agent's competence, we should consider a measure that commensurates with the agent's pursuing goal [3]. To determine a team's efficiency, which in fact demonstrates the degree of the soccer agent's effectiveness, the game result or the two teams score difference can be the preferred approach. To compare the three mentioned methods, three teams were set up accordingly. To diminish the effect of accidental results, the fuzzy team was made to contest ten times with each non-fuzzy one. As table 1 shows, the results remarkably confirm the fuzzy method's superiority. In order to measure the accuracy of different actions 10 matches for each of the three Nexus teams played with three other teams. The result is shown in Fig. 1 using the "SoccerDoctor" software [7] which is one of the best soccer simulation contest analyzers.

TABLE 1. THE RESULT OF COMPETITION BETWEEN THREE NEXUS TEAMS

3 Our 3D environment approaches

Because of the simplified model of 2D simulation league, a three-dimensional physical simulation was created. The three-dimensional physical simulator used in Soccer Simulation League addresses additional classes of problems as well as global team behavior, decision making procedures and etc.

Nexus team proposed a new scoring module [8] to select the best point on the goal line to shoot, considering player's position, catching and shooting time difference, and distance to target. To find the best point on the goal line to shoot, it is necessary to evaluate all points and obtain the one with the maximum calculated priority. Consequently we designed an algorithm which firstly eliminates the points at which ball can not reach due to opponent interception.

As a rule of thumb, the shoot evaluation module deals with physical aspects of the ball controller agent, opponents, goalie, and the ball. The aim is to find the best point on the goal line that if the ball is kicked based on which information; it will pass the goalkeeper ending inside the goal.

One of the parameters we need for the evaluation module is the temporal difference between ball and the goalie movement to reach the target. In other words, we calculate if the goalie reaches the target point sooner than the ball. This parameter would be then fed into the next fuzzy phase to estimate the catch probability. To do so we subtract the time take the agent to shoot considering rotation, from the time takes the goalie to reach the point and catch the ball. This subtraction trivially shows whether the ball pass the goalie or being intercepted. Let $T_b$ be the time takes ball to meet the target with the maximum speed, and $T_r$ be the rotation time for the ball controller to adjust it's position beside the ball. $T_g$ represents the time takes goalie to catch the ball (Fig. 2). Having calculated the above three parameters we define $\Delta t$ as:

$$\Delta t = T_g - (T_b + T_r)$$

If $\Delta t > 0$ then the ball would definitely pass the goalie and if $\Delta t < 0$ the ball would be intercepted. The greater $\Delta t$, the higher the probability of scoring goals. All these calculations were done assuming that there are no other agents except the goalkeeper in front of the ball controller to deviate the ball's direction. In order to approximate the physical features of the environment, 100 of offline training test cases in which an agent shoots the ball from certain point toward goal were done and results saved on a log file. Having saved the above data, we try to formulate Tb, Tg, and Tr by means of interpolation. The Gaussian function Tb(d) calculates the time takes the ball to pass distance d. Candidate shooting targets is a set of 25 points distributed along the goal line with 30cm interval. Fig. 3 shows temporal difference measurement (∆t) through the goal line.

In [8], we proposed a fuzzy approach to select best shoot decision. It has shown that fuzzy systems provide a simple, efficient, and fast way of decision-making in comparison with the cumbersome and tedious process of applying many different rules for achieving the same results. We expected the fuzzy system to be appropriate for shoot evaluation process in the soccer simulation environment, considering the noise produced by the soccer server and uncertainties which affect all the perceptions and actions of the agents. Our fuzzy rule base includes 15 rules.

The proposed algorithm was implemented on Nexus soccer simulation team. To measure the shoot performance, precision measure was used as the ratio of the number of goal retrieved to the number of shoots through the goal expressed as a percentage. As table 2 shows, the results of 50 shoots comparing fuzzy approach and the non-fuzzy one, confirm the proposed method's superiority.

TABLE 2. THE RESULTS OF 100 SHOOTS

4 Our 3D humanoid approaches

The current development of 3D Soccer Simulation League leads towards humanoid robots, which already can be controlled by a lower level interface. However, controllers for these robots have to be developed in order to provide an easy-to-use interface. The rules maturated in many points and gained focus on the issues that are essential from a technical point of view. Thus, the center of mass of all robots has to be on a certain height in relation to the size of the feet. Fundamental for playing soccer are the abilities to walk and to kick. As body contact between the physical agents is unavoidable, the capability of getting up after a fall is also essential. For keeping a goal, the robot must be able to perform special motions.

4.1 Walking skill

Delivering the weight from one leg to the other, shortening of the leg not needed for support, and leg motion in walking direction are the key ingredients of this gait. Walking forward, to the side, and rotating on the spot are generated in a similar way. The three basic walking directions can be smoothly combined. The robots are able to walk in every direction. Our Soccerbot agent tries to keep its center of mass (COM) at the same height across each step. We take advantage of COM implementation in ODE (Open Dynamic Engine). Currently the point of reference must correspond to the body's center of mass.

4.2 Kicking skill

After inhibiting the walking behavior and stopping, the robot moves its weight to the non-kicking leg and then shortens the kicking leg, swings it back and accelerates forward. The kicking leg reaches its maximal speed when it comes to the front of the robot.

4.3 Goalie dive skill

The goalie is capable of diving into both directions. First, it moves its COM and turns its upper body towards the left while shortening the legs. As soon as it tips over its left foot, it starts straightening its body again. While doing so it is sliding on its hands and elbows.

Acknowledgments

Authors would like to thank previous members of Nexus, the Ferdowsi University RoboCup soccer simulation team, for their modest cooperation.

References

[1] H.i Kitano, Y. Kuniyoshi, I. Noda, M. Asada, H. Matsubara, and E. Osawa. RoboCup: A challenge problem for AI. AI Magazine, 18(1):73–85, Spring 1997. [2] M. Chen, E. Foroughi,, Robocup Soccer Server manual 7.07, August, 2002. RoboCup Federation. available: http://sserver.sourceforge.net. [3] S. J. Russell, P. Norvig, Artificial Intelligence A modern Approach, Prentice Hall, 1995. [4] V. Salmani, M. Naghibzadeh, F. Seifi, and A. Taherinia, "A Two-Phase Mechanism for Agent's Action Selection in Soccer Simulation", The Second World Enformatika Conference, WEC'05, Istanbul, Turkey, pp. 217-220, February 2005. [5] V. Salmani, A. Milani Fard, M. Naghibzadeh, "A Fuzzy Two-Phase Decision Making Approach for Simulated Soccer Agent", IEEE International Conference on Engineering of Intelligent Systems, pp. 134-139, Islamabad, Pakistan, April 22-23 2006 [6] B. Kosko, Neural Networks and Fuzzy Systems, 1992, Englewood Cliffs: Prentice Hall [7] SoccerDoctor v1.0, developed by Wright Eagle, 2002. Available at: http://wrighteagle.org/sim/tools/SoccerDoctor.exe [8] M. Mozafari, A. Milani Fard, V. Salmani, M. Naghibzadeh, "An Improved Fuzzy Mechanism for 3D Soccer Player Agent's Shoot Skill", IEEE INDICON 2006, September 15-17, 2006, New Delhi, India