Alice Robocup Simulation 2D Team Description
Deng Yong, Xiao Zhou, Siying Chen, Zhengxi Gan
Gan HeFei Normal University
Abstract The paper mainly introduces the attack and defensive model. Our team use log Mining from reg and rel files to find better attack tactic and defensive tactic, this helps us design our team model more effectively. Besides, we research the chain action and use Monte carlo tree search to find best action chain, and finally we successfully build new evaluation function whit chain action.
Key words: Evaluation Model, Monte carlo tree search, Voroni Diagram, Mark strategy
1.Introduction
Our team was founded in 2016. This is the first time when we apply for the chance to participate in Robocup 2d competition. Our team code is based on agent2d3.1.1[1] which is released by Mr. Akiyama ,it also includes fragments of released code of Marlik2012[2] and Helios2008[3]. We sincerely appreciate the assistance from Hedehisa Akiyama and all of the relater open source developers.
2.Offensive Model
Our offensive model is divided into a ball kick strategy and a without ball strategy. Kick strategy includes an action set {dribble,pass,shoot,hold},The chain action model generates many different actions, holders use evaluation function to give different chain action a point and select the action chain with highest point to execute.
2.1 the evaluation model with chain action
We make a linear model to evaluate kick action, and use machine learning algorithm to get a better weight to suit different state.
E(action,state)=w1*a1+w2*a2+w3*a3+....wn*an
Current World Model state and the prediction of its development decide which action kicker will be selected. The kicker in a real world would consider many factors.For example,who is the opponent nearest to me, whether my pass target player can received ball safe or not,how to realize the target dist opponent goaland so on. We find action space and state space is so large ,and it makes function become too hard to suit every state situation. So we still try to find a better way to solute it .
2.2 Monte carlo tree search with chain action
Agent2d base code generates chain action as two depth when the second action is shoot. Also the base code evaluate model just considers the first step action. This makes player can't choose best chain action , if the second action is very good to attack. So we add generate chain action as two depth when the second action is shootand pass, and we also change base code evaluate model and make it can evaluate the second action of chain action.When we add chain action depth and evaluation model search depth, action and state space is too large, so we use Monte carlo tree search algorithm to search best chain action,then we use average evaluation to compare it.
E(average)=(E(first_action)+E(second_action)+..E(n))/chain_length
2.3 offensive move without ball
After data mining from much log, we find that our player's move position are very important when them is in penalty area. A good position always means a successful pass and shoot. If our player just move to the position of formation, it would be a bad point in most cases. Because the point of formation is static,in most cases our player would be marked by opponents. The vertex in Voronoi Diagram are equally spaced from three adjacent sites. We can put opponent's positions as input, then use Voronoi Diagram generation algorithm to calculate vertex of Voronoi Diagram as a set S. When our player's home position in opponent's penalty area, we will build an evaluation function to evaluate the point of set S and point is also in penalty area.
$$E=d1w1+d2w2+can_shoot_bonus*w3$$
Where E means evaluation score, d1 means the point dist from our formation point, d2 means the point dist from our player self, can_shoot_bonus means bonus's value if our player can shoot in the point. w1, w2 and w3 means weight and we use neural network to train and get the best weight.
3.Defensive Model
Our defensive model include {block,mark,formation} model,we find defensive skill is very important to against opponent's attacking.
3.1 New Marking Strategy
Many marking strategies have some problems, sometimes there will be two players marking the same opponent or no player marking a opponent who is in dangerous position. Because many marking strategies just consider one step, if a opponent player who is closet to me and I am closer to this opponent player than our team mates at the same time then I will mark the opponent. In order to solve these problems, we build this new marking strategy. We build an evaluation function to evaluate the opponent's marking value from our players:
$$s = d1 * w1 + d2 * w2$$
Where s means score,d1 means the opponent dist from our goal, d2 means the opponent dist from our player self, w1 and w2 means weight, we use neural network to train and get best weight. Our player use this function to evaluate opponent's marking value and then sort them, then our player will get his marking aim opponent if he judge himself is the closestone to the opponent player from opponent set which have been sorted in turn.
4 .Log Mining From game logs
We use python script to get kick data and move data from rcg and rcl files, and analysis this data, We get many tactic data from data mining from logs ,this help us design our team model more effectively.
5.Conclusion
This paper describe the research focus and current effort of Alice. Log mining provides us more clear message about how to design pass line and dribble line and so on. We still focus on research application of Voronoi Diagram, it's important for player's offensive move and can make our team more aggressive and we still strengthen our man mark strategy. In the future, we will spend more time on deep learning about agent, and try to use log mining to get more data to train deep neural network.