Arman 2006 3D Team Description
Mosalam Ebrahimi, Seyed Koosha Golmohammadi, Mohammad Ali Darvish Darab
Abstract This paper describes the new features of the Arman 3D soccer simulation team. Researches in our team were focused on two main categories -low level skills and decision making. We are using evolutionary fuzzy logic control algorithms for low-level control skills. We introduced a novel action selection model using learning Fuzzy Cognitive Maps (FCMs) for autonomous robots and used it in soccer agents.
Introduction
As the goal of Arman 3D simulation team has been researching, developing and implementing new methods in the field of Artificial Intelligence we tried some new algorithms for RoboCup 2006 on our team. Therefore we targeted two weakest points in our team, decision making and path tracking. We used evolutionary fuzzy logic control algorithms for path tracking as a main low level skill. For decision support of our agents we used our novel model for action selection in autonomous robots. This model is based on learning Fuzzy Cognitive Maps (FCMs) Fuzzy Cognitive Maps (FCMs) are well known intelligent analysts [3] because of their simplicity and transparency while being successful in a variety of applications. We adapted Nonlinear Hebbian Algorithm [2] to improve the FCM structure. In this paper we report our recent experiments. After very first tests, the simulation results were really encouraging.
Path Tracking Control
Since RoboCup Soccer Simulation is a real-time and dynamic domain using even a simulation step is critical. Our team suffered from weakness of steering control and this affected our team performance. We discovered evolutionary fuzzy controls are powerful path tracking controls [1]. This motivated us to use evolutionary fuzzy logic controls. This algorithm is introduced below.
Our algorithm finds the shortest segmented straight lines path between the agent and the ball. Then it utilizes a fuzzy path tracking controller that is trained by genetic programming to control the direction of the velocity.
The path tracker to be learned by GP is a two input, single output fuzzy controller that will map the error states into a proper steering angle at each time step. A population of candidate solutions is created from which a solution will emerge. The allowance for rule bases of various sizes enhances the diversity of the population. That is, the GP system creates individuals in the initial population that each have possibly different numbers of rules within a finite range (15-30) specified before a run. In the process of learning fuzzy control rules and membership functions, GP manipulates the linguistic variables directly associated with the controller. Given a desired motion behavior, the search space is contained in the set of all possible rule bases that can be composed recursively from a set of functions and a set of terminals. The function set consists of membership function definitions (describing controller inputs), components of the generic fuzzy if-then rule, and common fuzzy logic connectives. The terminal set is made up of the input and output linguistic variables and the corresponding membership functions associated with the problem.
The fitness function is a measure of performance used to rank each individual relative to others in the population. We compute path tracking performance by summing the Euclidean norms (normalized) of the final error states plus the average control effort ( ) over eight fitness cases. Thus, the following fitness function drives the evolution process:
Raw Fitness = $$\sum_{i=1}^{8} \sqrt{(\varepsilon_d^2 + \varepsilon_\theta^2 + \overline{\delta}^2)_i}$$
Where and are the position error and orientation error existing at the end of each fitness case simulation. The objective of this fitness function is to minimize final path tracking errors as well as the control effort expended. As such, a perfect fitness score is zero and, in general, lower fitness values are associated with better controllers.
Proposed Action Selection Model
We used learning FCMs for an optimal action selection. The agent is able to autonomously decide and respond to its changing environment. It chooses from actions on a simple basis. We considered 3 levels for states in our FCM model. Input parameters will affect the overall scores of other states in the second and third level. One state of the third level at a time will achieve the highest score which consequently lead to choose that state as the best action among all possible actions. For each individual action selection an optimal weighted matrix of the three level states is used. This matrix is developed based on an adapted Nonlinear Hebbian Algorithm [2].
The preliminary weights for matrices and concepts' values are proposed by expert. The agent is capable of updating these original weights based on an adaptive learning algorithm. for each iteration the matrix of weights and values of second and third states are extracted from the updated values of last iteration. Obviously the values of states in first level are specified through the environment.
In the following we illustrate how our model works. We described the flow of procedures in a striker as an agent in Soccer Simulation 3D environment in Fig.1. For the sake of simplicity we defined two input parameters in first level and two possible actions in third level. The striker agent could choose one of actions depending on game situations each time. We defined the weights and concept values as initial values.
In our model the agent responds to almost all changes in its environment. Generally in soccer possible situations are divided into two major categories for players. Category "A" includes all situations that the player does not posses the ball. Category "B" is the time that the player is in possession of the ball which we are interested in. The action selection in category "A" is mainly focused on changing the agent's location (position) in the field. However the action selection process in category "B" is mainly focused on what the agent should do with the ball.
The value of each state in FCM model is fuzzy. The fuzzy value ranges between 0 and 1 and is mapped with a "mapping function". In our example at any given time the mapping function of the distance of a striker agent in category "B" (in possession of the ball) to the opponent's goal is defined as $S_d$ where d is the actual distance. $S_d$ is defined as equations (2, 3):
$$S_d = 1 - \frac{d}{100}$$
If d> 100 THEN $$S_d = 0$$
This function maps agent's distance from opponent's goal and $S_d$ is the fuzzified value for its relative state.
In the proposed model Number of teammates in pass receiving situation is a parameter which results value of relative concept in first level. Fuzzified value for this concept is defined as $S_n$ . (In our model) n is defined according to three major elements at any given time; where 1 is the distance of teammate to the opponent's goal, 2 is the number of opponent agents between the striker agent and the opponent's goal, and finally 3 the distance of teammate to the striker agent. "n" is substituted in equations (4, 5) to evaluate $S_n$ :
$$S_n = \frac{n \times 2}{10}$$
If $$n > 5$$ THEN $S_n = 1$
The values of second and third level states and weight matrix which are updated with adapted nonlinear Hebbian algorithm lead to choose the highest scored state as desired action. However $S_d$ and $S_n$ are specified through environment and don't change in the learning process for each action.
Conclusion and Future Work
In this paper we summarized our experiments and researches on our 3D team for RoboCup 2006. We used evolutionary fuzzy logic control algorithms for low-level control skills. After implementation of the idea to our low-level skills in soccer agents very first test was encouraging. We used our new model for action selection of agents. The proposed model for action selection based on FCM gives the capability of selecting the best action among possible actions to autonomous robots. We implemented the suggested model to our own ARMAN 3D Soccer Simulation RoboCup team which was ranked 5th in RoboCup 2005. We tested both modified Arman and original Arman against Aria which ranked 1st in RoboCup 2005. The overall results were very promising.
References
- Zilouchian, A. and Jamshidi, M.: Intelligent Control Systems Using Soft Computing Methodologies. CRC Press LLC (2001)
- S. J. Li, R. M. Shen: Fuzzy Cognitive Map Learning Based On Improved Nonlinear Hebbian Rule. In proceedings of the Third International Conference on Machine Learning and Cybernetics, Shanghai (2004)
- K. Perusich, M. D. McNeese: Using Fuzzy Cognitive Maps as an Intelligent Analyst. IEEE International Conference on Computational Intelligence for Homeland Security and Personal Safety Orlando, FL, USA (2005)