MRL Research Proposal

Ashkan Ferdowsi

Qazvin Azad University – Mechatronic Research Labratory

Abstract Regarding to robocup goal in 2050, soccer simulation agents get closer to humanoid robots each year. Developing such robot in soccer simulation league includes two phases: implementation of low level and high level skills. The primary motivation behind the implementation of high level skills is to create a platform for research into simulated humanoid robot low level skills. In this research proposal we will describe some of low level skills and methods to implement them. Then we will proceed to some advanced skills, based on primary low level ones. Also experiments outcome from previous model (spheres) will be commented. We are going to import some of them that are implementable in humanoid model.

1 Introduction

Multi agent systems are environments deal with behavior management involved by several independent entities or agents, witch have the capability to communicating with each other and making decision based on their knowledge.

Body gestures are of tremendous importance for human-computer interaction, humanoid robotics[13] platform that consists of a humanoid robot. For controlling a humanoid robot [14] there are several methods that video processing -instead of motion capture- and some machine learning methods such as artificial neural networks[15], Reinforcement learning and evolutionary computing algorithms are common ones. Working on middle level and high level skills, requires efficient low level ones. Low level skill depends on some pre-tasks such as noise calibration, obtaining physical particles like acceleration and friction, mapping parameters to the Cartesian coordination, prediction, etc.

2 Software Engineering

Considering that rcssserver3d is a real-time and multi-agent environment , and everything is unavoidable, a well designed architecture of software could be one of the mainstay for each team. And it is because of increasing speed of processes and decreasing maintenance costs.

According to above we decide to design such a suitable architecture, by using many software engineering patterns and also innovation. We have used a hierarchical structure such that every layer can control its sub layers. Considering that there are many methods for working with humanoid robot, And also being component based, eventuates the ability of replacing every component (skills, utilities, etc) with anther one simple and soon.

3 Ball Estimation

Ball handling as one of the main issue for soccer agents is essential to implement various behaviors. So estimate of ball position in any states is very useful. Ball prediction will be used in many skills such as ball intercepting, pass, kick and so on. We will plan to use curve-fitting method for this purpose.

We will log some parameters of ball movements and then estimate the future status of ball by this method. Also, we will use kinematics methods for reaching ball movement equation. With these mechanisms, we will obtain some formulas about ball movements in air and ground states which is general for different situations. Estimation of ball status includes position, velocity and acceleration results in implementation of several useful functions. For example, time estimation of ball to reach certain position or predict position of ball after specific time and so on. Therefore forecasting status of ball is very considerable.

4 Noise Reduction

Agent localization and ball tracking are the significant topics in soccer environments. So reducing noise of sensed data is very important. According to our pervious agents [16], we are going to use Kalman filter [17] as a well-known and widely used method for reducing noise (but not in this version). We will use this method to achieve accurate localization. We will utilize two types of filtering:

DKF (Discrete Kalman Filtering): whereas ball movements aren't so complex, we will want to use DKF for recognizing the position of ball in static and moving states.

EKF (Extended Kalman Filtering): agent localization and in particular self-localization will be done with EKF. In our pervious work, we reduced average of localization error to 0.04m and we will hope to reach it in this version.

5 Artificial Neural Network in Low Level Skills

We are interested in using neural network for finding true value of rotation of legs for walking. By observing humanoid robot walking, we figure out that first step is finding suitable values of difference bending angles between thigh and knee during variants velocities.

As figure implies there is a symmetric between differences of angles for legs. Next step is using neural network for mapping differences to input velocities. We found out that by changing velocity of both legs there are various values for differences of angles as shown in figure 2.

So using multi layer preceptor, we can calculate relationship between angular rate input for legs and their angles. After that we can simply calculate which angle is sufficient for walking by trial and error.

Other issue is kicking that can be solved with neural network, so as pervious experience in soccer 3D agent we can use the same technique here by changing some parameter while the framework remains fixed.

By kicking the ball with different velocities of legs we can calculate which angular velocity is suitable. And also we can predict the maximum distance of ball from kick point with this velocity.

6 High Level Skills

Coordination among agents following same goal in a dynamic uncertain environment is a difficult task. In such complex environments agents should find and follow a strategy which accomplishes to goal.

In previous version (spheres model) of Robocop soccer simulation server we were working on games strategies. We're going to import and develop our above mention methods in the humanoid simulation server. To obtain this goal we should develop our high level management into two phases consists of developing a planner and generating set of patterns of the planner with the neural network and evolutionary computing methods.

6.1 Planning

Coordination among agents following same goal in a dynamic uncertain environment is a difficult task. In such complex environments agents should find and follow a strategy which accomplishes to goal. In this way a planner is needed that can deal with complexity of environment and determine agents' tasks and the way of executing them.

Market-based techniques are one of the best approaches and are more responsive than other techniques to change, but they need high message passing ability. Taking advantages of market-based architecture, we introduce a novel approach called Pattern-Based Planning System (PBPS) that can be used in environments with forcible limitations on message passing.

Supplying four inputs, PBPS generates a dynamic plan as output that determines agents' tasks and sequence of execution of those tasks to achieve the goal. The first one is definition of the initial state. The second is the description of tasks that agents can perform. The third is a set of objective functions used to evaluate different tasks and plans to indicate how they direct team toward goal. And the last input that's specified to our approach is a set of patterns designed by a human expert. Finding best path from root to a leaf in PatternsTree a plan is generated.

There are several advantages with our Pattern-Based planning system. Implementing patterns decreases search space of finding the optimum sequence of tasks and is fast and effective to coordinate agents regarding to team's goal. It also causes to decrease message passing among agents and becomes more important in environments that communication is highly limited. On the other hand PBPS is flexible to get implemented in competitive environments where agents should overcome opponents.

We implemented our planner in "MRL soccer 3D" team's agents (sphere model) that the simulation results show that our approach increase probability of achieving the goal. Also if low-level skills on humanoid model which are input of planner be created correctly, we are able to implement PBPS on new model too. Moreover PBPS includes of controller that monitors and adapts plan during its execution. Importing architecture of this component, we are designing and implementing a controller for low-level skills.

6.2 Coordination optimization using Evolutionary Computing and Neural Network

In last version of Robocop soccer simulation server we were working on team tactics. We hope to import and develop this work to the new humanoid model. According to the ball position we determined three game states: defense, offense and midfield game states. Our methods are divided into two main sections: off-line and on-line parts using neural network in order to recognize and cluster opponents' playing styles and also evolutionary computing algorithms in order to obtain some suitable team tactics for each one of them in every game state.

In off-line mode we define some playing styles for each game state and run different opponents against our team. The neural network adds another playing style if the opponent is playing more different from our predefined game styles. Then EC methods generate some team tactics containing targets that each agent should achieve. In this way some adapted team tactics for the related opponent play style regarding to game state is returned.

In on-line mode we use Adaptive Resonance Theory 2-A (ART2-A) neural network for recognizing playing style of soccer agent that has high speed for categorizing and recognition. The vigilance parameter in these networks has considerable influence on this system and should be chosen properly. Higher value causes highly detailed memories while lower value result in more general memories.

References

1. A. Pongpunwattana, R. Rysdyk, J. Vagners, D. Rarthbun. Market-based co-evalution planning for multiple autonomous vehicles. University of Washington (2003)
2. A. Pongpunwattana. Real-Time Planning for Teams of Autonomous Vehicles in Dynamic Uncertain Environments. PH.D. thesis, University of Washington. (2004)
3. M. Bowling, B. Browning and M. Veloso. Plays as team plans for coordination and adaptation. In Proceedings of the 14th International Conference on Automated Planning and Scheduling (ICAPS-04), Vancouver. (2004)
4. O. Obst, A. Maas, and J. Boedecker. HTN Planning for Flexible Coordination Of Multiagent Team Behavior. In Proceedings of the IJCAI-05 Workshop on Agents in Real-Time and Dynamic Environments, Edinburgh, Scotland. (2005) 87–94
5. N. Kalyviotis and H. Hu. A co-operative framework for strategic planning. In proceeding of the 3rd British Conference on Autonomous Mobile Robotics & Autonomous System, Manchester. (2001)
6. L. E. Parker. On the design of behavior-based multi-robot teams. Journal of Advanced Robotics. 10(6) (1996)
7. L. Parker, "Alliance: An architecture for fault tolerant multi-robot cooperation," IEEE Transactions on Robotics and Automation, 14(2) (1998) 220–240
8. J. T. Feddema, C. Lewis, and D. A. Schoenwald. Decentralized control of cooperative robotic vehicles: Theory and application. IEEE Transactions on Robotics and Automation. 18(2002) 852–864
9. R. G. Smith. The contract net protocol: High-level communication and control in a distributed problem solver. IEEE Transactions on Computers. 10 (1980) 1104–1113
10. M. P. Wellman. Market-oriented programming: Some early lessons. in Market-Based Control: A Paradigm for Distributed Resource Allocation (S. Clearwater, ed.),ch. 4, World Scientific, (1996)
11. M. P. Wellman and P. R. Wurman. Market-aware agents for a multiagent world. Robotics and Autonomous Systems, 24 (1998) 115–125
12. M. B. Dias and A. Stentz. A free market architecture for distributed control of a multirobot system. In Proceedings of the 6th International conference on intelligent autonomous systems, (2000) 115–122
13. Mathias Kolsch & Craig Martell Towards a Common Human Gesture Description Language Department of Computer Science, Naval Postgraduate School, Monterey, CA 93943
14. Daniel Joseph Paluska Design of a Humanoid Biped for Walking Research MASSACHUSETTS INSTITUTE OF TECHNOLOGY
15. Artificial Neural Networks A Tutorial Jain (Ieee 1996).
16. Mahmood Rahmani, S.Mohammad H.S.Javadi, Ashkan Ferdowsi, Mahdi N.Moghadasi, Siavash Aslani, , Nima Shafii., MRL Soccer 3D Team Description, Accepted in RoboCup 2006, Bremen,German.
17. 1. R. E. Kalman. "A New Approach to Linear Filtering and Prediction Problems", Transaction of the ASME-Journal of Basic Engineering, pp. 35-45 (1960)