Persepolis Soccer 2D Simulation Team Description Paper 2021

Morteza Noohpisheh, Maryam Shekarriz, Fatemeh Zaremehrjardi, Farnaz Khademi Ardekani, Seyedeh Afsoon Khorsand

Dept. of Computer Engineering, Shiraz University, Iran; Dept. of Computer Science, St. Francis Xavier University; Dept. of Biomedical Engineering, Kazeroon University; Dept. of Industrial Management, Zand University

Abstract This paper includes some explanations about algorithms implemented by Persepolis team members. We will introduce algorithms that are used for pass, shoot, dribbling and marking, and in particular with the ranking algorithm for the states of chain action. The base code used by Persepolis is agent-2d 3.1.1[17].

1 Introduction

The goal is to implement intelligent agents, that have the ability to decide on using soccer individual skills as well as the ability to work with other agents in the soccer field. In addition, programming has been done by agents to improve individual skills (shoot, pass, dribble, positions of players on the field). Razi team is formed from students, has started its activities from November 2012 and has made plan in order to achieve its goal, participating in World Cup competitions. This team has participated in prestigious competitions and has won the third place in IranOpen2017 and IranOpen2018. This team has begun to use artificial intelligence in its decision making. It continues its work with the aim of using artificial intelligence algorithms and online analysis.

Note : The name of this team was changed from Razi to Persepolis after the Japan Open 2020.

2 Activities Done in Other Iranian Teams

In the recent years, the following tasks have been done in other Iranian teams: A formation 2 detection system as well as a software called Tournament Planning and Analyzing Software(TPAS) have been developed by Namira[5][6][7]. Dynamic formation changing, field evaluation system for offense, field evaluation system optimization, Optimizing defensive decision making by using message passing between agents, Shooting behavior optimization using Deep Neural Network, Predicting opponent's behavior using Rough Neural Network, Predicting the behavior of the agents with ball possession, Defensive Decision Making using Deep Reinforcement Learning have been developed by Cyrus[8][9][10][11][13]. Nexus has implemented Reinforcement Learning for decision making in penalty area[14].

3 Previous Works in Team

2018: Razi has organized its own system for rating states in chain action[20]. 2019: Use the Gravity Strategy[21].

4 Offensive Strategy

In the 2D soccer simulation league, the field state has many properties describing offence and defense, such as the relative positions, the distance from the ball, the members in a certain region and so on. It is easy to see that the field state is a continuous space. To deal with the state data quickly and effectively, we need to divide them into many equivalent states by a process of discretization. In the course of a soccer game, the attacking strategy varies with the state. That is to say, attacking strategy is a piecewise function: the best attack action may be different if the field state changes. Furthermore, we must identify the best action for each state. In this strategy we set a path for each action in each state, and found the best action in a given state via the preferences of the paths. We should regard each attack action as a path in the solution space, consider each agent's training in the same way we would foraging behavior of an ant, and treat an effective attack as success foraging. In order to solve the problem of "a solution per state", we set an ant group for each equivalent state. Within this setting, each ant lays down a trail pheromones on a path (corresponding to the preference value of an attack action), and each ant group may have a path that has the highest pheromone level (corresponding to the best preference value and the best attack action).

4.1 Offensive strategy algorithm

We can simulate different attacking environment by setting agents at different points on the field. Regarding three attack actions (shooting, passing and dribbling) as three different paths for foraging, we can obtain the best attack action by using the preference values of ants' foraging behaviors. The following two steps should be made in order to find the best attack action: 1) train the preference values of actions in each environment, and 2) select the best attack action based on preference value. The first step runs offline in the background for a long time to obtain the preference value; the second step is designed using the decision module of the attacker and runs online to select the best action. The state of attacking environment is a continuous space. In order to deal with the state data quickly and effectively, we should transfer them into many discrete equivalent states. For every attacking environment (state), we generated a group of ants and used pheromone concentration to record the dynamic success-rate of each attack action.

5 Experimental evaluation

After being created, the 2D soccer simulation team with the MACO[16] was run in the RoboCup simulation platform to evaluate our new algorithm. Firstly, every agent took random attack actions. The results of these actions have been recorded and used for the next training session. The underlying code and underlying database are Agent2d-3.1.1 and Librcsc, respectively. The positions of all players (two attackers, two defenders and one goalkeeper) were set randomly by an offline coach. After five cycles of competition, the results of two teams are recorded as the format of "αshoot, dshoot, αpass, dintp, ddrib, dintd, aa, suc". The training will stop after 150,000 times. The training scene is shown in Fig. 1. In order to show the situations of the attack and defense sides more clearly, only the penalty area is depicted.

5.1 Competitions with baseline strategy

Application effect in competition is the sole criterion for evaluating an offensive strategy. This section concerns our creation of a 2D soccer team named Persepolis, which employs the MACO algorithm as the offensive strategy. Meanwhile, using the underlying code Agent2d-3.1.1, which is an open source by Helios (former world champion in the RoboCup 2D simulation league), we created another team, which adopted the searching-space offensive strategy. The only difference between the Persepolis team and the BASE team is the attacking selection strategy. There were 10 games in total in the test, with each game running 6,000 standard clock cycles. The results of the games are presented below: As listed in Table 1, the Persepolis team dominated in the games with 53.4% possession and 82.61% passing-success on average. As a result, the Persepolis team enjoyed a 100% winning advantage with 3.4 goal differences per game, as well as scoring three times as many goals as its opponent on average.

Table 1.

6 Conclusions

The offensive strategy is an important factor in the attacking capability of a soccer team. Given the unpredictability of the environment and the influence amongst agents, it is very difficult to select action for the ball-handler. Attacking strategy has becomes a hotly discussed issue in the domain of multi-agent cooperation in a real-time, asynchronous and noisy environment. We analyzed the principal factors of the main attack actions for the ball-handler in the 2D simulation league and presented a novel offensive strategy algorithm based on multi-group ant colony optimization (MACO-OS) in order to improve attacking efficiency. Using the pheromone evaporation mechanism, the algorithm creates an AIT, which is the basis of decision-making action. Our future work will focus on defensive strategy and defense conversion strategy to enhance the team's overall performance.

References

1. Frieder Stolzenburg, OliverObst, Jan Murry, Qualitative Velocity and ball Interception, 2002
2. Hausknecht, Matthew, and Peter Stone. "Deep reinforcement learning in parameterized action space." arXiv preprint arXiv:1511.04143 (2015)
3. Gabel, T.,Riedmiller, M.Trost, F.:A Case Study on Improving Defense Behavior in Soccer Simulation 2D:The NeuroHassle Approach.In:Springer-Verlag Berlin Heidelberg (2009) 61-72 An Introduction to Genetic Algorithms , Jenna Carr , May 16, 2014
4. "RoboCup Soccer Simulation", http://en.sourceforge.jp/projects/rctools, 2017
5. E. Asali, F. Negahbani, S. Tafazzol, M. S. Maghareh, S. Bahmeie, S. Barazandeh, S. Mirian and M. Moshkelgosha, "Namira Soccer 2D Simulation Team Description Paper 2018," in RoboCup 2018, Monreal, Canada, 2018.
6. E. Asali, A. Moravej, S. Akbarpoor, O. Asali, M. Katebzadeh, S. Tafazol, F. Negahbani, M. Valipour, S. Mirian, H. Mehran, A. Bitaraf Haghighi and Y. Khoshghadam Borazjani, "Persian Gulf Soccer 2D Simulation Team Description Paper 2017," in RoboCup 2017, Nagoya, Japan, 2017.
7. E. Asali, M. Valipour, N. Zare, A. Afshar, M. Katebzadeh and G. Dastghaibyfard, "Using Machine Learning approaches to detect opponent formation," Artificial Intelligence and Robotics (IRANOPEN), pp. 140-144, 2016.
8. N. Zare, M. Sadeghipour, A. Keshavarzi, M. Sarvmaili, A. Nikanjam, R. Aghayari, A. Firouzkoohi, M. Abolnejad, S. Elahimanesh and A. Akhgari, "Cyrus 2D Simulation Team Description Paper 2018," in RoboCup 2018, Montreal, Canada, 2018.
9. N. Zare, A. Najimi, M. Sarvmaili, A. Akbarpour, M. NaghipourFar, B. Barahimi and A. Nikanjam, "CYRUS 2D Simulation Team Description Paper 2017," in 21th annual RoboCup International Symposium, Nagoya, Japan, 2017.
10. R. Khayami, N. Zare, M. Karimi, P. Mahor, A. Afshar, M. S. Najafi, M. Asadi, F. Tekrar, E. Asali and A. Keshavarzi, "CYRUS 2D simulation team description paper 2014," in RoboCup 2014 Symposium and Competitions: Team description papers, Joao Pessoa, 2014.
11. R. Khayami, N. Zare, H. Zolanvar, M. Karimi, P. Mahoor, F. Tekrar, E. Asali and M. Fatehi, "Cyrus Soccer 2D Sumilation Team Description Paper," in 17th annual RoboCup International Symposium, Eindhoven, Netherlands, 2013.
12. M. Hausknecht and P. Stone, "Deep Reinforcement Learning in Parameterized Action Space," in International Conference on Learning Representations (ICLR), San Juan, Puerto Rico, 2016.
13. Nader Zare , Mahtab Sarvmaili , Omid Mehrabian , Amin Nikanjam , Seyed Hossein Khasteh , Aref Sayareh, Omid Amini, Borna Barahimi, Arshia Majidi, Aria Mostajeran , "CYRUS 2D Simulation Team Description Paper 2019"
14. M. Abolfazli Esfahani, M. Ghafouri, M. Jamili, S. Askari, R. Etemadi, H. Mohebbi, M. Dara, M. Erafani and M. Malekabadi, "Nexus2D Team Description Paper," in RoboCup 2017, Nagoya, Japan, 2017.
15. Genetic Algorithm Optimized GravityBased RoboCup Soccer Team ,Tory Harter,July 28, 2015
16. Chen, S., Lv, G., Wang, X. (2016). Offensive Strategy in the 2D Soccer Simulation League Using Multi-Group Ant Colony Optimization. International Journal of Advanced Robotic Systems, 13.
17. Akiyama H, Nakashima T. Helios base: An open source package for the robocup soccer 2d simulation. InRobot Soccer World Cup 2013 Jun 24 (pp. 528-535). Springer, Berlin, Heidelberg.
18. Fukushima T, Nakashima T, Akiyama H. Similarity Analysis of Action Trajectories Based on Kick Distributions. InRobot World Cup 2019 Jul 2 (pp. 58-70). Springer, Cham.
19. Akiyama, H., Nakashima, T., Fukushima, T., Suzuki, Y., Ohori, A.: Helios2019: Team description paper. In: RoboCup 2019 Symposium and Competitions, Sydney, Australia (2019)
20. M. Noohpisheh, M. Shekarriz, S. Barzegar, D. Borzoo, A. Kariminia, "Razi Soccer 2D Simulation Team Description Paper 2018," in RoboCup 2018, Montreal, Canada, 2018.
21. Noohpisheh M, Shekarriz M, Bordbar A, Liaghat M, Salimi A, Borzoo D, Zarei A. M. Noohpisheh, M. Shekarriz, A. Bordbar, D. Borzoo, M. Liaghat, A. Salimi, A. Zarei, "Razi Soccer 2D Simulation Team Description Paper 2019," in RoboCup 2019, Sydney, Australia, 2019.