View Original PDF

Damash 2013 Soccer 2D Simulation Team Description Paper

Reza Vatani, Pooyan Piredeir, Sepideh Rahmatinia, Sepand Haghighi, Seyedeh Reyhaneh Kamali

University Of Guilan – UGM-Lab, Rasht, I.R. Iran

Abstract Damash is a new innovator team which is working to be a leading team in presenting new optimization ideas in 2D Soccer Simulation using Artificial Intelligence and Machine Learning Methods. As Damash's primary ideas in RoboCup 2013, an optimum pass skill based on Ant Colony Optimization and a novel defense skill mechanism based on Neural Network is introduced and explained in this paper.

Introduction

Damash is a new 2D Soccer Simulation Project by three former members of Marlik 2D Soccer Simulation team [1] and two other members. According to our feedbacks, An Attempt is felt for good results in Iranian teams in RoboCup competitions. This attempt made a gap between us and leading academic works in 2D Soccer Simulation League. In the shadow of this gap, Due to applying our novel ideas and algorithms based on our new studies and academic projects we stated a new team based on innovation, Machine Learning and advanced Artificial Intelligence research to present new and leading ideas in 2D Soccer Simulation League.

2D Soccer Simulation as a Multi-Agent environment is used for developing our research, trying to prepare Damash, which is based on Agent2D [2], as platform for applying AI algorithms and more similarities to real football game.

In This Paper we describe Damash's new method for making flexible pass skill based on Ant Colony Optimization Algorithm (MFPS-ACO) and a novel Neural Network based Functional Defense Skill Mechanism (NNFDSM), which we are working on them in order to make our team more flexible and adaptive in strategy during a game. In the other hand, in our new Damash project, it is tried to adapt our strategies correctly not only before a game using AI algorithms and Machine Learning [3].

Flexible Pass Skill Based on Ant Colony Optimization

According to the experiences gained from developing three kinds of pass skill in Marlik 2D Soccer Simulation Team [4], pass skill is done intelligently in Damash. When the pass is executed for the ball owner, there are some choices in teammates. An innovative idea is inspired from Ant Colony Optimization Algorithm in our new pass skill to have more flexible performance versus other teams in a game [5].

In this pass skill method, each agent has ten teammates as choices to kick the ball toward and make pass. Some of these choices are omitted by the prediction function after predicting that the choice wouldn't catch the ball. Henceforth, each remained choice is checked and rated by factors that were applied on Marlik [4]. In Damash, a different method based on Roulette Wheel Selection is used after checking and ranking each choice [6]. Choice and path's ranks are normalized and transformed to probabilistic individual. Then by using Roulette Wheel Strategies, one of the choices and paths are chosen randomly and pass is executed by the ball owner to the chosen agent. By using say ability and the prediction function for estimating ball catch time, the pass executer agent notices its correct pass and statically trails some pheromone with the number of pass catcher teammate [7].

Remained Choices after Appling Pass Prediction Function.
Remained Choices after Appling Pass Prediction Function.
Ratably Pheromone Trailing.
Ratably Pheromone Trailing.

Neural Network based Functional Defense Skill Mechanism

In Damash, we are working on a new mechanism for evaluating probability of goal acceptation based on Back Propagation Neural Network. BPNN is a multi-layer network with a non-linear transfer function and Widrow-Hoff learning rule. Input vector and targets used to train this network and approximate a function or estimate probability and etc. As respect that back propagation neural network consist of sigmoid layers (in this case log-sigmoid) have this ability to approximate any function with finite discontinuity points.

In Damash, we want to peruse goal acceptation probability by using of several neural networks from view of different defenders [8].

Input Parameters are:

  1. Defender Distance to Gate Center
  2. Ball Distance to Gate Center
  3. Defenders X-Position Variance
  4. Defenders Y-Position Variance
  5. Manhattan Distance Between 2Centeral Defenders

And output is logical classification form, (0,1), means goal acceptation or not.

Network Architecture is Feed-Forward and Probability is evaluated separately and in parallel by each defender. To achieve high-speed converge we used Levenberg–Marquardt algorithm that used for non-linear function and is stronger than Gauss-Newton and Gradient-Descent.

Cost Function that should be minimized:

$$S(p) = f^{T} f = \sum_{i=1}^{m} [f_{i}(p)]^{2}$$

In each iteration p vector replaced by a new approximation and J is Jacobian of f at p:

$$f(p+q) \approx f(p) + Jq$$

According to $\nabla_q S = 0$ :

$$(J^T J)q = -J^T f$$

And the weights are updated as follows:

$$w(k+1) = w(k) - (J^{T}J + \lambda I)^{-1}J^{T}\varepsilon(k)$$

[9]

First network trained and initialized by NNtools in Matlab, then Pre-Trained Function train online by coach.

And coach update player weights in Pauses.

If during the game average of probability be bigger than threshold then with some change in function parameter and player position will try to avoid goal acceptation.

Goal Acceptation Probability in 200 First Cycles, Damash vs GDUT Tiji
Goal Acceptation Probability in 200 First Cycles, Damash vs GDUT Tiji

Conclusions and Future Work

Two new ideas were described in this paper as our work on Damash. We hope to make these ideas more efficient and applicable in near future. There is still a lot to do on this team but we depend on our new ideas and academic works.

In our next projects, we want to describe a new Neural Network Based Penalties. Also maintaining described ideas would be our future programs.

Acknowledgment

This team is grateful to University of Guilan Mechatronics Laboratory (UGM-Lab) and Marlik Institute of Innovation and Technology for their supports. We would like to acknowledge Helios team for their source releases and contributions to 2D community. We also would like to acknowledge Prof. Geoffrey E. Hinton for helping us and his great online courses.

References

  1. Tavafi, A., Nozari, N., Vatani, R., Rad Yousefi, M., Rahmatinia, S., Pirdir, P.: Marlik 2012 Team Description. RoboCup 2012 Symposium and Competitions, MexicoCity, 2012.
  2. Akiyama, H., http://rctools.sourceforge.jp
  3. Marian, S., Luca, D., Sarac, B., Cotarlea, O.: OXSY 2012 Team Description. RoboCup 2012 Symposium and Competitions, MexicoCity, 2012.
  4. Tavafi, A., Nozari, N., Vatani, R., Rad Yousefi, M., Rahmatinia, S., Piredeir, P.: Marlik 2011 Team Description. RoboCup 2011 Symposium and Competitions, Istanbul, 2011.
  5. Dorigo, M., Optimization, Learning and Natural Algorithms, PhD thesis, Politecnico di Milano, Italy, 1992.
  6. Zhong, J., Hu, X., Gu, M., Zhang, J., Comparison of Performance between Different Selection Strategies on Simple Genetic Algorithms, 2005 International Conference on Computational Intelligence for Modelling, Control and Automation, and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC 05), 28-30 Nov. 2005, Vienna, Austria
  7. M. Dorigo, V. Maniezzo, et A. Colorni, Ant system: optimization by a colony of cooperating agents, IEEE Transactions on Systems, Man, and Cybernetics-Part B , volume 26, numéro 1, pages 29-41, 1996.
  8. Likas, A., Probability Density Estimation Using Artificial Neural Networks, Computer Physics Communications, Elsevier, 135 (2001), 167-175
  9. DOHNAL, J., Using of Levenberg-Marquardt Method in Identification by Neural Networks, Department of Control and Instrumentation, FEEC, BUT