RoboCupRescue 2010 - Robot League Team Hector Darmstadt (Germany)

Micha Andriluka, Stefan Kohlbrecher, Johannes Meyer, Karen Petersen, Paul Schnitzspan, Oskar von Stryk

Department of Computer Science, Technische Universität Darmstadt; Department of Mechanical Engineering, Technische Universität Darmstadt

www.gkmm.tu-darmstadt.de/rescue

Abstract The team Hector Darmstadt has been established from a PhD program funded by the German Research Foundation at TU Darmstadt. It combines expertise from Computer Science and Mechanical Engineering. In 2009 the team successfully participated for the first time in the RoboCup rescue league. Several team members have already contributed in the past to highly successful teams in the RoboCup fourlegged and humanoid leagues and in UAV competitions.

Introduction

The Team Hector Darmstadt (Heterogeneous Cooperating Team of Robots) has been established in late 2008 within the PhD program "Cooperative, Adaptive and Responsive Monitoring in Mixed Mode Environments" (Research Training Group GRK 1362, www.gkmm.tu-darmstadt.de) funded by the German Research Foundation (DFG). This program addresses two exciting and challenging research areas: (1) navigation and coordination of multiple autonomous vehicles to perform a common task possibly together with a human mission manager; and (2) monitoring in mixed mode environments that are characterized by the heterogeneity of their components in terms of resources, capabilities and connectivity. The participation in RoboCup Rescue is one of the first steps towards a heterogeneous real-world scenario. Driven by the goal of using heterogeneous cooperative hardware and software in disaster environments, a successful participation in RoboCup Rescue will be an important milestone for these efforts. The interdisciplinarity of our Research Training Group allows us to combine established knowledge and elaborate tools from different disciplines to develop new solutions in search and rescue applications in the long run.

The experience in hardware [1] and software [2] of autonomous robots has already been successfully applied to RoboCup soccer [3, 4], and there have been studies in simulation on cooperative control [5, 6]. The team participated successfully for the first time in RoboCup rescue 2009 [7]. Several members have contributed to two top teams in the Four-Legged League (the GermanTeam) and the Humanoid League (Darmstadt Dribblers). Other members of the group are developing different computer vision algorithms for people detection and object recognition [8, 9] which can be applied to the Search and Rescue scenario. In this group there is also a history of highly successful participation in recognition and perception challenges for computer vision. Finally, the team members from mechanical engineering are focusing on the design and experimental evaluation of unmanned aerial and ground vehicles for environmental monitoring and surveillance applications. They successfully participated in the flight competition of the European Micro Air Vehicle Conference (EMAV'09) and won the first prize in class outdoor autonomy.

Our ground robots are based on the R/C model car Kyosho Twin Force (later on referred to as "Hector GV", Fig. 1) The vehicles are modified for better autonomous handling and enhanced with an onboard computer and two laser range finders. For victim identification we developed a vision extension, including a visual and a thermal camera mounted on a pan/tilt unit. The control box can be used as a stand-alone component for testing or can be attached to another robot to enable autonomous exploration and victim detection.

Based on the experience from RoboCup 2009, several improvements have been made to the chassis of the robot. The steering now uses less connection rods and stronger digital servos, yielding more direct control of the steering angles even on rough terrain or when wheels are blocked for some reason.

1 Team Members and Their Contributions

– Karen Petersen: Team Leader, Behavior

– Johannes Meyer: Hardware Design, Navigation

– Paul Schnitzplan, Micha Andriluka and Konstantin Fuchs: Visual Object Recognition and Sensor Fusion

– Stefan Kohlbrecher: SLAM, GUI

– Peter Schauß: Navigation, Path Planning

– Peter Englert: GUI

– Yongkie Wiyogo: Elevation Mapping – Martin Friedmann: Simulation – Dirk Thomas: Software Framework

– Oskar von Stryk: Advisor

2 Operator Station Set-up and Break-Down (10 minutes)

Our system consists of the lightweight Hector GV that is able to work autonomously or can be remote controlled via a laptop or by radio control. The whole control equipment (even if we add a joystick or a gamepad) easily fits into a standard backpack and the Hector GV can be carried by hand. To start a mission, the robot and the laptop have to be switched on, and the operator can connect to the robot via Wireless LAN.

3 Communications

Our communication concept is based on three different channels. A common wireless network is used for high-bandwidth data like video images or map information. Currently we use a 2.4 GHz 802.11g/n network, but hardware also allows 5 GHz or 802.11a/n operation if necessary. For data exchange with lower bandwidth demands the vehicle is additionally equipped with an 802.15.4 compatible radio device. This link is used for telemetry and basic manual driving of the vehicle and enables the operator to take control even when the onboard computer is no longer operational. The operator station will be connected to a modified wireless access point which operates both the wireless network and the 802.15.4 link.

Communication channels used

4 Control Method and Human-Robot Interface

In general we focus more on autonomy than on mobility and manual control. So the ideal case is that the operator doesn't have to do anything during the mission besides monitoring what the robot is doing and confirming the victim information the robot provides. But anyway there are many ways how we can control the robot's behavior, and the operator can take over control at any time. Monitoring: The whole software system is based on RoboFrame [2]. It consist of two parts: (1) an Application (RoboApp) that is running on the robot, and (2) a Graphical User Interface (RoboGui) that is running on the operator's computer. Every behavior decision and all messages between software modules, including sensor readings, can be monitored by the RoboGui. In that way every action of the robot is transparent to the operator.

The internal world model of the robot can be visualized in a configurable dialog. The user can choose to display the robot's position, sensor readings, etc. Displayed camera images can be overlaid with further information like the output of the thermal camera or estimated victim position.

Adjustable Autonomy: To allow for flexible remote control of a robot, an interface is currently developed that supports control at different levels of autonomy and therefore is able to integrate several solutions for the same subtask with varying level of autonomy. To illustrate the idea of different levels of autonomy, consider the task of driving to a certain position. This can be either accomplished autonomously (by choosing the waypoint autonomously or by a klick on the map), or by following a predefined path (draw the path in the map), or by full remote control (use a joystick). The idea of adjustable autonomy can also be applied to other tasks like object detection or mapping.

5 Map Generation/Printing

The simultaneous localization and mapping (SLAM) problem is solved by using a grid map representation that gets updated using a combined probabilistic particle filter/scan matching approach. The particle filter is used to generate a starting estimate for the scan matcher, which then aligns the current scan with the existing map using a maximum likelihood approach based on gradient ascent. Our approach has low runtime requirements and can run with an update rate of 20–40 Hz while requiring less than 10% CPU time on a Core2Duo setup, thus freeing resources for other computation.

The input used for solving the SLAM problem are laser scans and the robot state as estimated by the navigation filter (cf. section 6). The data provided by the navigation filter is used to update the particle distribution using the motion model, as well as for transformation of laser scans to take into account the attitude of the laser scanner and vehicle during acquisition of scans. Figure 5 shows two maps learned using the described SLAM system.

To better negotiate the increasingly rough terrain in the rescue arena, we will use an elevation mapping approach using a second laser scanner tilted downwards to scan the ground in front of the vehicle. An elevation grid map is then created by using these measurements in conjunction with the pose estimate provided by the 2D SLAM module described above. This elevation map is then used to plan safe paths through the environment, taking into account non-flat underground such as ramps.

The map can get annotated manually or automatically with information about hazmat symbols and victims. It can be converted and saved in the Geo-TIFF format.

6 Sensors for Navigation and Localization

Wheel Encoders: To measure the translational and rotational speed of the vehicle, all four wheels are equipped with incremental optical encoders. This odometry data is used especially for indoor navigation, but due to the inaccuracy additional feedback from other sensors is needed.

Laser Scanners: The vehicle is equipped with two laser scanners: A tiltable Hokuyo URG04-LX scanner is mounted in the front of the vehicle. It is mainly intended for scanning the ground in front of the vehicle. The second laser scanner, a Hokuyo UTM30-LX, is mounted on a roll/tilt unit at the front of the control box and is mainly used for 2D mapping. Both scanners can be stabilized to stay close to their intended scan plane regardless of vehicle attitude.

Ultrasound Range Finders: Additionally to the laser scanners, a set of ultrasound range finders mounted at the back of the vehicle enables autonomous reactive collision avoidance when moving backwards, as the scanners only cover 270 degrees field of view.

Inertial Measurement Unit: To measure the attitude of the vehicle, it is equipped with a 6DoF inertial sensor ADIS16350 by Analog Devices which measures accelerations and angular rates.

Navigation filter: All sensor information is fused to get an overall estimate of position, velocity and attitude of the vehicle by using an extended Kalman filter. Although Kalman filtering is a common and simple approach for robot navigation problems, it suffers amongst others from the resulting unimodal representation of the belief state. On the other side, the feedback from map-based localization, as described in section 5, can lead to ambiguities which contradict the Gaussian assumption. Our approach is to combine these two sources of information in a loosely-coupled way in order to gain a robust navigation solution. The attitude estimate of the navigation filter is used to stabilize the laser scanners and camera system.

7 Sensors for Victim Identification

Victim detection will be approached from several complementary directions. With team members working in computer vision we are able to leverage their extensive experience and prior work. Significant progress in visual object recognition and scene understanding now allows us to apply these methods to real-life conditions. The victim detection will be supported by integrating other sensor types, like a thermal camera and laser range scanner.

Vision-Based Recognition of Victims and Hazmat Symbols: The recognition of the objects is performed by using a combination of visual cues based on the gradients of image intensity. Such cues can be efficiently captured by a descriptor based on the histograms of oriented gradients (see Fig. 3 for illustration). First, the gradient magnitude and orientation are computed densely in the image. The local distributions of the gradient orientation are then captured by the histogram. Such histograms are then grouped with their neighbors and jointly normalized. The normalization and local pooling of gradient information significantly improves the stability of the description to viewpoint changes, noise and changes in illumination.

It has been demonstrated that visual information represented in this way combined with powerful machine learning techniques can be successfully applied to recognition of people in realistic conditions [10]. While showing good performance this approach also requires significant processing power. The on-board computer (Fig. 4) with an nVidia graphics card allows real-time feature computation and recognition with an implementation based on [11].

We use the recognition system for detection of hazmat symbols at the victim sites (Fig. 4). The same system, but trained on the images of human body parts, is used to recognize victims parts.

Multi-Cue Victim Detection: In addition to visual victim detection we will use a thermal camera as our secondary sensor. Thermal images often contain not only victims but also other warm objects, such as radiators or fire, so that thermal and visual recognition systems will deliver complementary information. This complementary information can be used to increase the reliability of our framework since victim hypotheses in thermal images can be verified with our visual inference method and vice versa. With this sensor fusion scheme many false alarms retrieved by single sensor systems can be avoided.

8 Robot Locomotion

Our vehicle is based on a Kyosho Twin Force RC model with a powerful and fast drive train. For indoor navigation we modified the drive train, the steering and the suspension because of the much higher weight.

4-wheel-drive: The 4-wheel-drive of the vehicle has one differential gear per axis and no middle differential gear. This ensures that the vehicle is able to move when only a portion of wheels have ground contact. To reduce the maximum speed for indoor operation and to increase the torque we added a 1:5 gear.

4-wheel-steering: The front and rear wheels can be controlled independently, providing three advantages over normal 2-wheel-steering: (1) a smaller minimum turn radius (half of 2-wheel-steering), (2) the possibility that the rear wheels use the same trajectory as the front wheels (if both steering angles are the same) (3) the possibility to move sidewards (up to 35 degrees to the longitudinal axis of the vehicle).

Normally the rear wheels are set to the same steering angle as the front wheels, so that the resulting trajectories are identical and the risk of obstacle contact is reduced. With this vehicle we have a very flexible, mobile and powerful platform which additionally has the advantage of providing precise odometry information.

9 Other Mechanisms

This section describes established technologies from RoboCup experience and hardware modularity concepts used in the Hector GV.

9.1 Established Technologies from RoboCup Experience

From 2001 till 2008 the Darmstadt Dribbling Dackels participated in the 4-legged soccer league as a part of the German Team and won the world championship in 2004, 2005 and 2008. Since 2004 the Darmstadt Dribblers participate successfully in the humanoid kid-size league and won the world championship and the award for the best humanoid robot in 2009. Although Search and Rescue is a totally different application than soccer, the team Hector Darmstadt can make use of the experiences from the soccer teams and many tools that were developed in these teams can also be applied for Search and Rescue.

RoboFrame: Our software is based on RoboFrame [2], a software framework that supports teams of heterogeneous autonomous lightweight robots. RoboFrame supports modular software development and takes care of the communication between sensors, actuators and software modules.

XABSL: The high-level behavior is described as a hierarchical state machine with the Extensible Agent Behavior Specification Language XABSL [12]. This allows to easily extend the behavior and to reuse existing parts in different contexts. XABSL was originally developed for the behavior of soccer robots, but it was also applied to team cooperation of heterogeneous robots [13].

MuRoSimF: The Multi-Robot-Simulation-Framework [14] provides components for the simulation of a robot's motion and sensing capabilities with different levels of detail. MuRoSimF allows to test each component of the software separately by replacing all other part by ground truth data. After component testing, before using the real hardware, the whole system can either be tested in a MuRoSimF based simulation or with the established simulator USARsim [15].

9.2 Hardware Modularity

The complete hardware structure of our vehicle is shown in Fig.5. The intrinsic sensors and actuators are connected to an interface board which communicates with a PC/104 computer. This setup alone enables 6DoF navigation and allows basic autonomous driving. Most of the extrinsic sensors are connected to a separate on-board computer which is equipped with a state-of-the-art Core 2 Duo mobile CPU and a high-performance GPU for parallel computing and fulfills the more demanding tasks of mapping and visual detection of victims and hazmat symbols.

This separation of both components, even on hardware layer, simplifies independent testing and offers a high degree of flexibility. The vision extension can easily be mounted on other robots or used as separate instrument for the evaluation of computer vision algorithms. The robot itself is used in various outdoor scenarios as fast and lightweight research platform.

10 Team Training for Operation (Human Factors)

Our User Interface, RoboGui, is based on the software framework RoboFrame [2], which is a very general framework which enables the development of modular software, that can easily be ported to other platforms. But due to this generality, the User Interface cannot be specialized to a specific system, which leads to the fact that a new user needs some time for getting familiar with the system. However, currently a more specialized interface is developed, that presents all relevant data clearly arranged to the operator and allows the operator to interact with the robot intuitively.

11 Possibility for Practical Application to Real Disaster Site

The Hector GV is a fast vehicle that allows for precise and versatile locomotion. The low weight is a big advantage for fast and flexible setup of the whole system. The most critical points are movement in very rough terrain and sensitivity against some basic environmental factors like humidity.

The strength of our approach is the elaborate reusable software [2], which is a reliable base for developing and extending our system. For practical application to real disaster sites we have to improve abilities in (partial) autonomy and plan to combine the system with other existing systems like an UAV (Quadrotor) and (mobile) sensor nodes. We hope to be able to give useful, flexible assistance to operators in managing disaster scenario within a few years.

12 System Cost

The system costs are broken down into vehicle components and vision extension components.

12.1 Vehicle

The vehicle component costs are detailed in the following table.

Vehicle Components

12.2 Vision Extension

The vision extension components and their costs are listed in the following table.

Vision Extension Components

References

1. M. Friedmann, S. Petters, M. Risler, H. Sakamoto, D. Thomas, and O. von Stryk. New autonomous, four-legged and humanoid robots for research and education. In Workshop Proceedings of the Intl. Conf. on Simulation, Modeling and Programming for Autonomous Robots, pages 570–579, Venice (Italy), November 3-4 2008.
2. S. Petters, D. Thomas, and O. von Stryk. Roboframe a modular software framework for lightweight autonomous robots. In Proc. Workshop on Measures and Procedures for the Evaluation of Robot Architectures and Middleware of the 2007 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2007.
3. M. Friedmann, K. Petersen, S. Petters, K. Radkhah, D. Thomas, and O. von Stryk. Darmstadt dribblers: Team description for humanoid kidsize league of robocup 2008. Technical report, Technische Universität Darmstadt, 2008.
4. D. Becker, J. Brose, D. Göhring, M. Jüngel, M. Risler, and T. Röfer. Germanteam 2008 - the german national robocup team. Technical report, DFKI Bremen, TU Darmstadt, HU Berlin, 2008.
5. Christian Reinl and Oskar von Stryk. Optimal control of multi-vehicle systems under communication constraints using mixed-integer linear programming. In Proc. of the. 1st Intl. Conf. on Robot Communication and Coordination (RoboComm), Athens, Greece, Oct. 15-17 2007. ICST.
6. K.Listmann, M. Masalawala, and J. Adamy. Consensus for formation control of nonholonomic mobile robots. In Proc. of the IEEE Intl. Conf. on Robotics and Automation (accepted for publication), 2009.
7. Micha Andriluka, Martin Friedmann, Stefan Kohlbrecher, Johannes Meyer, Karen Petersen, Christian Reinl, Paul Schnitzspan Peter Schauß and, Armin Strobel, Dirk Thomas, Anguelina Vatcheva, and Oskar von Stryk. Robocuprescue 2009 - robot league team darmstadt rescue robot team (germany). Technical report, Technische Universität Darmstadt, 2009.
8. M. Andriluka, S. Roth, and B. Schiele. People-tracking-by-detection and people-detection-by-tracking. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'08), 2008.
9. P. Schnitzspan, M. Fritz, and B. Schiele. Hierarchical support vector random fields: Joint training to combine local and global features. In Europoean Conf. on Computer Vision (ECCV 2008), October, 2008.
10. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'05), 2005.
11. C. Wojek, G. Dorkó, A. Schulz, and B. Schiele. Sliding-windows for rapid object class localization: A parallel technique. In DAGM-Symposium, pages 71–81, 2008.
12. M. Lötzsch, M. Risler, and M. Jüngel. XABSL a pragmatic approach to behavior engineering. In Proc. of IEEE/RSJ Intl. Conf. of Intelligent Robots and Systems (IROS), pages 5124–5129, Beijing, China, 2006.
13. J. Kiener and O. von Stryk. Cooperation of heterogeneous, autonomous robots: A case study of humanoid and wheeled robots. In Proc. IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), pages 959–964, 2007.
14. M. Friedmann, K. Petersen, and O. von Stryk. Simulation of multi-robot teams with flexible level of detail. In Simulation, Modeling and Programming for Autonomous Robots (SIMPAR 2008), pages 29–40, Venice, Italy, November 2008.
15. S. Balakirsky, C. Scrapper, S. Carpin, and M. Lewis. Usarsim: providing a framework for multi-robot performance evaluation. In Proceedings of PerMIS, 2006.