View Original PDF

NaoTH 2011

Hans-Dieter Burkhard, Thomas Krause, Heinrich Mellmann, Claas-Norman Ritter, Yuan Xu, Marcus Scheunemann, Martin Schneider, Florian Holzhauer

Institut f¨ur Informatik, LFG K¨unstliche Intelligenz, Humboldt-Universit¨at zu Berlin, Rudower Chaussee 25, 12489 Berlin, Germany

http://www.naoth.de · http://www.fumanoids.de/

Abstract The research group Nao-Team Humboldt was founded at the end of 2007 and is part of the AI research Lab of the Humboldt-Universität, with 2 PhD students and around 10 Master Students in the core team. The team has a long tradition in RoboCup, previously winning the Four Legged league three times. They have participated in both the Standard Platform League and 3D Simulation League with a common code base, achieving multiple top placements including 2nd place at the RoboCup World Championship 2010. Their general research fields include agent oriented techniques and machine learning with applications to cognitive robotics, with current focus on: narrowing the gap between simulated and real robots, software architecture for autonomous agents, dynamic motion control, and world modeling.

1 Introduction

The research group Nao-Team Humboldt was founded at the end of 2007 and consists of students and researchers at the Humboldt-Universit¨at zu Berlin, coming from a number of different countries during the last years, in particular from Germany, Italy, Russia, Egypt, China, Iraq, and Iran. The team is a part of the AI research Lab of the Humboldt-Universit¨at which is headed by Prof. Dr. Hans-Dieter Burkhard and is led by Heinrich Mellmann. At the current state we have 2 Phd students and around 10 Master Students in the core team. Additionally we provide courses and seminars where the students solve tasks related to RoboCup. We have a long tradition within the RoboCup by working for the Four Legged league as a part of the GermanTeam in recent years, where we won the competition three times.

The current team members are: Heinrich Mellmann, Yuan Xu, Thomas Krause, Claas-Norman Ritter, Florian Holzhauer, Marcus Scheunemann, Paul Sch¨utte, Martin Schneider, Kirill Yasinovskiy, Luisa Jahn, Dominik Krienelke, Christian Rekittke.

We started with Naos in May 2008 and achieved the 4. place at the competition in Suzhou 2008. We achieved 3. place in technical challenge at Graz 2009. In 2010, we won the 2. place at RomeCup/Rom, 1. Place in Athens/Greece, and 4. place at German Open 2010. In 2010 we first time participated simultaneously in SPL in Simulation 3D with the same code. In the 3D Simulation we won the German Open and the AutCup competitions and achieved the 2. place at the RoboCup World Championship 2010 in Singapore. In 2011 we won the Iran Open competition in SPL and got 4. place in Simulation 3D, at the German Open we got 4. place as well.

With our efforts in both leagues, we hope to foster the cooperation between the two leagues and to improve both of them. We also cooperate with several other teams and associate projects at different universities, e.g., we regularly make test games with the humanoid team FUManoids1 .

1 http://www.fumanoids.de/

Our general research fields include agent oriented techniques and machine learning with their applications to cognitive robotics. As our record shows, our results are recognized outside RoboCup as well. Our current research focuses among others mainly on the following topics:

  • Narrowing the gap between simulated and real robots (section 2)
  • Software architecture for an autonomous agent (section 3)
  • Dynamic motion control (section 4)
  • World modeling (section 5)

We will described them briefly in the next sections, please refer to our recent publications [1–7] for more details.

2 Simulation and Real Robots

As a common experience, there are big gaps between simulation and reality in robotics, especially with regards to basic physics with consequences for low level skills in motion and perception. There are some researchers who have already tried to narrow this gap, but there are only few successful results so far. We investigate the relationships and possibilities for methods and code transferring. Consequences can lead to better simulation tools, especially in the 3D Simulation League. At the moment, we participate in both, Standard Platform League and 3D Simulation League with the common core of our program. As already stated, therewith, we want to foster the cooperation between the two leagues and to improve both of them.

When compared to real Nao robots, some devices are missing in the SimSpark, as LEDs and sound speakers. On one hand, we extended the standard version of SimSpark by adding missing devices like camera, accelerometer, to simulate the real robot. On the other hand, we can use a virtual vision sensor which is used in 3D simulation league instead of our image processing module. This allows us to perform isolated experiments on low level (e.g., image processing) and also on high level (e.g., team behavior). Also we developed a common architecture [2], and published a simple framework allowing for an easy start in the Simulation 3D league.

Our plan is to analyze data from sensors/actuators in simulation and from real robots at first and then to apply machine learning methods to improve the available model or build a good implicit model from the data of real robot. Particularly, we plan to:

  • improve the simulated model of the robot in SimSpark;
  • publish the architecture and a version of SimSpark which can be used for simulation in SPL;
  • transfer results from simulation to the real robot (e.g., team behavior, navigation with potential field);

So far, we have developed walking gaits through evolutionary techniques in a simulated environment [8, 9]. Reinforcememt Learning was used for the development of dribbling skills in the 2D simulation [10], while Case Based Reasoning was used for strategic behavior [11, 12]. BDI-techniques have been investigated for behavior control, e.g., in [13, 14].

3 Architecture

An appropriate architecture, i.e., framework, is the base of each successful heterogeneous software project. It enables a group of developers to work at the same project and to organize their solutions. From this point of view, the artificial intelligence and/or robotics related research projects are usually more complicated, since the actual result of the project is often not clear. In particular, a strong organization of the software is necessary if the project is involved in education. Our software architecture is organized with the main focus on modularity, easy usage, transparency and easy testing. Here is a short overview of some most important parts of our project; please refer to our recent publication [2] for more details.

platform interface is an abstraction layer which allows for a separation between the platform independent part and platform specific one. The platform independent part, called the Core, contains the actual implementation of all cognitive processes whereas the specific one is responsible for the communication with a particular platform, e.g., Simspark, Nao etc.. Thus, we are able to run the same code on different platforms without changing the actual algorithms.

module architecture is a framework based on a blackboard architecture which is used to organize the workflow of the cognitive/deliberative part of the program. The modules contain particular implementations of cognitive algorithms, e.g., ball detection, self localization, planning etc.. The communication between the modules is realized through the blackboard. Thus, it is easy to enable/disable modules during the runtime, e.g., if there are different modules for the same task or some experimental modules which shouldn't be used in a game, and to monitor the data flow.

debug infrastructure allows to transfer any debug information from/to the robot during the runtime. Based on this infrastructure we implemented a number of debugging concepts like switching on/off some debugging code, plotting, measuring execution time, graphical visualization in 2D and 3D and some more. Debug results are transferred over the network and monitored / visualized by our debugging tools which are able to monitor the state of a single robot as well as a whole team.

communication between the robot and a PC is essential for debugging purposes and controlling tasks. It is designed in a very generic way, i.e., it makes as few assumptions about the availability of some given debugging software GUI or the existence of all possible external helper libraries. Even with very few assumptions about the kind of communication and the used software stack we want to give the developers a powerful and flexible communication framework for interacting with the robot. In a minimal setup it is possible to perform all the debugging/monitoring operations using telnet.

testing is essential for a large and heterogeneous project. Our team has a large code base which is being developed by a changing set of team members with various levels of education and knowledge about the project. Tests can be run by each developer after code changes to prevent undesired side-effects or broken code. The tests are implemented on two different granularity levels: Unit-Tests — calling a specific function with sample input, and Integration Tests — tests of a module to ensure that a module follows the requirements of our module framework and works properly.

Upon a commit in our code repository, an automated build system compiles and tests the committed source code. If the build or the tests fail, the committer is notified by email about the error. As a side effect, the tests can be used as examples helping new developers to understand the parts of our framework faster and easier.

4 Dynamic Motion Control

The performance of a soccer robot is highly dependent on its motion ability. Together with the ability to walk, the kicking motion is one of the most important motions in a soccer game. However, at the current state the most common approaches of implementing the kick are based on key frame technique. Such solutions are inflexible and costs a lot of time to adjust robot's position. Moreover, they are hard to integrate into the general motion flow, e.g., for the change between walk and kick the robot has usually to change to a special stand position.

Fixed motions such as keyframe nets perform well in a very restricted way and determinate environments. More flexible motions must be able to adapt to different conditions. There are at least two specifications: Adaption to control demands, e.g., required changes of speed and direction, omnidirectional walk, and adaptation to the environment, e.g., different floors. The adaptation of the kick according to the ball state and fluent change between walk and kick are another examples.

At the current state we have a stable version of an omnidirectional walk control and a dynamic kick which are used in the games. Along with further improvements of the dynamic walk and kick motions our current research focuses in particular on integration of the motions, e.g., fluent change between walk and kick.

Adaptation to changing conditions requires feedback from sensors. We experiment with the different sensors of the NAO. Especially, adaptation to the visual data, e.g., seen ball or optical flow, is investigated. Problems arise from sensor noise and delays within the feedback loop. Within a correlated project we also investigate the paradigm of local control loops, e.g., we extended the Nao with additional sensors.

We have investigated different Optimization and Machine Learning approaches, as evolutionary approaches [15, 8] and recurrent Neural Networks [16] for Aibos and humanoid robots, and - as mentioned earlier - Reinforcement Learning (RL) with applications for the 2D simulation league [10]. Recently, another PhD thesis with focus on RL for humanoid robot walking is being written.

4.1 Dynamic Kick

For the implementation of the dynamic kick motion we have to consider following aspects:

  • planning the motion according to the seen ball and the desired kick direction
  • reachability space of the motion
  • stability while standing on one foot, and moving the other

Together with the desired direction and the strength of the kick this aspects define conditions which have to be satisfied by the motion trajectory in order to perform the kick. In our approach we generate the motion trajectory dynamically according to this conditions, which allows the robot to handle different situations and adapt to changing conditions, e.g., moving ball.

We implement and test our approach in simulation and on the real robot. At the current state the robot is able to kick the ball from any position which is reachable by the foot to any suitable direction. Our current research concerns amongst others, kicking of a moving ball and dynamically change between walk and kick, i.e., without stop.

For detailed description of the implementation, please refer to [5, 6]. Videos showing some experiments performed on the real robot can be found on our homepage.

5 Perception and World Modeling

In order to realize a complex and successful cooperative behavior it is necessary to have a appropriate model of the surrounding world. Thus, one of the main focuses of our current research is the improvement of the perceptional abilities of the robot and its capabilities to build a world model.

5.1 SURF based Object Recognition and Tracking

In order to get more independent from color coded image processing we are actively working on using standard CV approaches in the SPL. One of these approaches is the SURF feature detection as provided by the OpenCV library. We integrated OpenCV into our code base and try to learn features to recognize objects, such as the ball, goal posts and robots. Since SURF only needs a gray scale image and is quite tolerant to light changes, this approach minimizes the requirement of calibration before games. One must also find ways to aggregate the detected features into more complex models of objects (like robots). Currently we are able to detect other robots including their rotation using distinctive features on the feet. Future research will focus on classification (learning) and tracking of the features.

We already achieved a processing speed of about 20ms on the real robot. With special pre-filtering and selection of the areas which should be considered for the SURF feature detection we want to improve the runtime of the algorithm even further.

5.2 Constraint Selflocalization

We investigate the application of Constraint based techniques for navigation, and compare the approach to classical Bayesian approaches like Kalman filters and Monte-Carlo methods. Kalman filters make certain assumptions about the environment (linearity of the model, Gaussian noise), Monte-Carlo methods as particle filters are restricted by high computational demand and thus, low dimensionality. We have investigated these methods under various perspectives in the previous years [17–24] including missing information [25]. Constraint techniques have to handle inconsistent data, but can be advantageous whenever ambiguous data is available [26]. They are computationally cheap using interval arithmetic, and they can be easily communicated allowing for cooperative localization [27– 32].

Classical approaches are prone to error propagation (e.g. from camera matrix over self localization to global positions of other objects). Using constraints, all related parameters (e.g. kinematic chain, camera matrix, positions) can be connected by related constraints and the common solution can be calculated by propagation techniques, [33–35].

Constraints are built by image percepts from flags, goals, lines and other objects (even dynamical ones like ball and other players) [29]. They are able to deal with missing information [25] as well. Other sensors such as joint sensors lead to constraints for the camera matrix. Further constraints can use communicated information and past information, e.g., object speed. Each information can be used to constrain the related parameters. The shape of the constraint is determined by the type of information and the expected sensor noise. We use constraint propagation (interval arithmetic) for further restrictions of possible values of the parameters, as for positions [33, 35].

Recent investigations and experiments deal with the treatment of inconsistencies (e.g. by ghost percepts) and further increase of performance. Furthermore we developed and tested a constraint-based player model.

References

  1. Cotugno, G., Mellmann, H.: Dynamic motion control: Adaptive bimanual grasping for a humanoid robot. In: Proceedings of the Workshop on Concurrency, Specification, and Programming CS&P 2010. Volume Volume 2., B¨ornicke (near Berlin), Germany (September 2010)
  2. Mellmann, H., Xu, Y., Krause, T., Holzhauer, F.: Naoth software architecture for an autonomous agent. In: International Workshop on Standards and Common Platforms for Robotics (SCPR 2010), Darmstadt (November 2010)
  3. Xu, Y., Mellmann, H., Burkhard, H.D.: An approach to close the gap between simulation and real robots. In Ando, Balakirska, Reggiani, von Stryk, eds.: 2nd International Conference on Simulation, Modeling and Programming for Autonomous Robots (SIMPAR), Darmstadt (November 2010)
  4. Xu, Y., Burkhard, H.D.: Narrowing reality gap and validation: Improving the simulator for humanoid soccer robot. In: Concurrency, Specification and Programming CS&P'2010, Helenenau, Germany (September 2010)
  5. Xu, Y., Mellmann, H.: Adaptive motion control: Dynamic kick for a humanoid robot. In: the Proceedings of the 33rd Annual German Conference on Artificial Intelligence, Karlsruhe, Germany (2010)
  6. Mellmann, H., Xu, Y.: Adaptive motion control with visual feedback for a humanoid robot. In: the Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei (2010)
  7. Mellmann, H.: Ein anderes Modell der Welt. Alternative Methoden zur Lokalisierung mobiler Roboter. Diploma thesis, Humboldt-Universit¨at zu Berlin, Institut f¨ur Informatik (2010)
  8. Hein, D., Hild, M., Berger, R.: Evolution of biped walking using neural oscillators and physical simulation. In: RoboCup 2007: Robot Soccer World Cup XI. Lecture Notes in Artificial Intelligence, Springer (2007)
  9. Hein, D.: Simloid evolution of biped walking using physical simulation. Diploma thesis, Humboldt-Universit¨at zu Berlin, Institut f¨ur Informatik (2007)
  10. Uc-Cetina, V.: Reinforcement Learning in Continuous State and Action Spaces. PhD thesis, Humboldt-Universit¨at zu Berlin (2009)
  11. Burkhard, H.D., Berger, R.: Cases in robotic soccer. In Rosina O. Weber, M.M.R., ed.: Case-Based Reasoning Research and Development, Proc. 7th International Conference on Case-Based Reasoning, ICCBR 2007. Lecture Notes in Artificial Intelligence, Springer (2007) 1–15
  12. Berger, R., L¨ammel, G.: Exploiting Past Experience. Case-Based Decision Support for Soccer Agents. In: Proceedings of the 30th Annual German Conference on Artificial Intelligence (KI'07), Springer (2007)
  13. Berger, R.: Die Doppelpass-Architektur. Verhaltenssteuerung autonomer Agenten in dynamischen Umgebungen (in German). Diploma thesis, Humboldt-Universit¨at zu Berlin, Institut f¨ur Informatik (2006)
  14. Burkhard, H.D.: Programming Bounded Rationality. In: Proceedings of the International Workshop on Monitoring, Security, and Rescue Techniques in Multiagent Systems (MSRAS 2004), Springer (2005) 347–362
  15. D¨uffert, U.: Vierbeiniges Laufen: Modellierung und Optimierung von Roboterbewegungen (in German) (2004) Diploma thesis.
  16. Hild, M.: Neurodynamische Module zur Bewegungssteuerung autonomer mobiler Roboter (in German). PhD thesis, Humboldt-Universit¨at zu Berlin (2007)
  17. Hoffmann, J.: Proprioceptive Motion Modeling for Monte Carlo Localization. In: RoboCup 2006: Robot Soccer World Cup X. Lecture Notes in Artificial Intelligence, Springer (2007)
  18. J¨ungel, M.: Bearing-only localization for mobile robots. In: Proceedings of the 2007 International Conference on Advanced Robotics (ICAR 2007), Jeju, Korea,. (August 2007)
  19. J¨ungel, M.: Self-localization based on a short-term memory of bearings and odometry. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), San Diego (October 2007)
  20. J¨ungel, M., Mellmann, H.: Memory-based state-estimation. Fundamenta Informaticae 85(Number 1-4) (2008) 297–311
  21. J¨ungel, M., Mellmann, H., Spranger, M.: Improving vision-based distance measurements using reference objects. In Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F., eds.: RoboCup 2007: Robot Soccer World Cup XI. Volume Volume 5001/2008 of Lecture Notes in Computer Science., Springer Berlin / Heidelberg (2007) 89–100
  22. J¨ungel, M., Risler, M.: Self-localization using odometry and horizontal bearings to landmarks. In Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F., eds.: RoboCup 2007: Robot Soccer World Cup XI. Lecture Notes in Artificial Intelligence, Springer (2007)
  23. Mellmann, H., J¨ungel, M., Spranger, M.: Using reference objects to improve visionbased bearing measurements. In: Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 2008, Acropolis Convention Center, Nice, France, IEEE (22–26 Sept. 2008) 3939–3945
  24. Mellmann, H.: Active landmark selection for vision-based self-localization. In: Proceedings of the Workshop on Concurrency, Specification, and Programming CS&P 2009. Volume Volume 2., Krak´ow-Przegorzaly, Poland (28–30 September 2009) 398–405
  25. Hoffmann, J., Spranger, M., G¨ohring, D., J¨ungel, M., Burkhard, H.D.: Further studies on the use of negative information. In: Proceedings of the 2006 IEEE International Conference on Robotics and Automation (ICRA) IEEE. (2006)
  26. G¨ohring, D., Mellmann, H., Gerasymova, K., Burkhard, H.D.: Constraint based world modeling. Fundamenta Informaticae 85(Number 1-4) (2008) 123–137
  27. G¨ohring, D.: Distributed object modeling using object relations in dynamic environments. In: Proceedings of Concurrency, Specification and Programming (CS&P 2006). (2006)
  28. G¨ohring, D., Hoffmann, J.: Sensor Modeling Using Visual-Object Relation in Multi Robot Object Tracking. In: RoboCup 2006: Robot Soccer World Cup X. Lecture Notes in Artificial Intelligence, Springer (2007)
  29. G¨ohring, D., Burkhard, H.D.: Multi robot object tracking and self localization using visual percept relations. In: Proceedings of IEEE/RSJ International Conference of Intelligent Robots and Systems (IROS), pages 31–36, IEEE (2006)
  30. G¨ohring, D.: Cooperative object localization using line-based percept communication. In Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F., eds.: RoboCup 2007: Robot Soccer World Cup XI. Lecture Notes in Artificial Intelligence, Springer (2007)
  31. G¨ohring, D., Burkhard, H.D.: Cooperative world modeling in dynamic multi-robot environments. Fundamenta Informaticae 75(1–4) (2007) 281–294
  32. G¨ohring, D.: Constraint Based World Modeling for Multi Agent Systems in Dynamic Environments. PhD thesis, Humboldt-Universit¨at zu Berlin (2009)
  33. G¨ohring, D., Mellmann, H., Burkhard, H.D.: Constraint based belief modeling. In Iocchi, L., Matsubara, H., Weitzenfeld, A., Zhou, C., eds.: RoboCup 2008: Robot Soccer World Cup XII. Lecture Notes in Artificial Intelligence, Springer (2008)
  34. G¨ohring, D., Mellmann, H., Burkhard, H.D.: Constraint based object state modeling. In Herman, B., Libor, P., Miroslav, K., eds.: European Robotics Symposium 2008. Volume Volume 44/2008 of Springer Tracts in Advanced Robotics., Prague, Chech Republic, Springer Berlin / Heidelberg (2008) 63–72 This volume (EUROS 2008).
  35. G¨ohring, D., Mellmann, H., Burkhard, H.D.: Constraint based world modeling in mobile robotics. In: Proc. IEEE International Conference on Robotics and Automation ICRA 2009. (2009) 2538–2543