RoboCupRescue 2011 - Robot League Team Hector Darmstadt (Germany)

Thorsten Graber, Stefan Kohlbrecher, Johannes Meyer, Karen Petersen, Oskar von Stryk

Department of Computer Science, Technische Universität Darmstadt; Department of Mechanical Engineering, Technische Universität Darmstadt

www.gkmm.tu-darmstadt.de/rescue

Abstract The team Hector Darmstadt has been established from a PhD program funded by the German Research Foundation at TU Darmstadt. It combines expertise from Computer Science and Mechanical Engineering. The team successfully participates in the RoboCup Rescue League since 2009, with a focus on autonomous robots. Several team members have already contributed in the past to highly successful teams in the RoboCup Four-Legged and Humanoid League and in UAV competitions.

Introduction

The Team Hector Darmstadt (Heterogeneous Cooperating Team of Robots) has been established in late 2008 within the PhD program "Cooperative, Adaptive and Responsive Monitoring in Mixed Mode Environments" (Research Training Group GRK 1362, www.gkmm.tu-darmstadt.de) funded by the German Research Foundation (DFG). This program addresses two exciting and challenging research areas: (1) navigation and coordination of multiple autonomous vehicles to perform a common task possibly together with a human mission manager; and (2) monitoring in mixed mode environments that are characterized by the heterogeneity of their components in terms of resources, capabilities and connectivity. The participation in RoboCup Rescue is one of the steps towards a heterogeneous real-world scenario. Driven by the goal of using heterogeneous cooperative hardware and software in disaster environments, a successful participation in RoboCup Rescue is an important milestone for these efforts. The interdisciplinarity of our Research Training Group allows us to combine established knowledge and elaborate tools from different disciplines to develop new solutions in search and rescue applications in the long run.

This research has been supported by the German Research Foundation (DFG) within the Research Training Group 1362 "Cooperative, adaptive and responsive monitoring in mixed mode environments"

The experience in hardware [1] and software [2] of autonomous robots has already been successfully applied to RoboCup Soccer [3, 4], and there have been studies in simulation on cooperative control [5, 6]. The team participated successfully for the first time in RoboCup Rescue 2009. Several members have contributed to two top teams in the Four-Legged League (the GermanTeam) and the Humanoid League (Darmstadt Dribblers). Other members of the group are developing different computer vision algorithms for people detection and object recognition [7, 8] which can be applied to the Search and Rescue scenario. In this group there is also a history of highly successful participation in recognition and perception challenges for computer vision. Finally, team members from mechanical engineering are focusing on the design and experimental evaluation of unmanned aerial and ground vehicles for environmental monitoring and surveillance applications. They successfully participated in the flight competition of the European Micro Air Vehicle Conference (EMAV'09) and won the first prize in class outdoor autonomy.

Our ground robots are based on the R/C model car Kyosho Twin Force (later on referred to as "Hector GV", Fig. 1) The vehicles are modified for better autonomous handling and enhanced with an onboard computer and two laser range finders. For victim identification we developed a vision extension, including a visual and a thermal camera mounted on a pan/tilt unit. The control box can be used as a stand-alone component for testing or can be attached to another robot to enable autonomous exploration and victim detection.

Based on the experience from previous RoboCup competitions we participated in, several improvements have been made to the chassis of the robot. Compared to the original design, the steering system uses less connection rods and stronger digital servos, yielding more direct control of the steering angles even on rough terrain or when wheels are blocked for some reason.

The major additions and changes compared to the system we used in 2010 are:

Usage of ROS (Robot Operating System) as middleware.
Deployment of multiple cooperative UGVs at the same time with map merging.
Usage of RGB-D sensors.

1 Team Members and Their Contributions

Stefan Kohlbrecher: Team Leader, SLAM, GUI
Karen Petersen: Behavior, HRI, Team Cooperation
Johannes Meyer: Hardware, Navigation and Control, Simulation
Thorsten Graber: Point Cloud Processing
Florian Kunz: ROS Software Integration/Infrastructure
Mark Sollweck: Exploration/Global Path Planning
Tomislav Hasan: Map Merging
Thomas Kanold: Local Path Planning
Martin Friedmann: Simulation
Oskar von Stryk: Advisor

2 Operator Station Set-up and Break-Down (10 minutes)

Our system consists of one or more lightweight Hector GVs capable of autonomous or remote controlled operation via a laptop. All of the control equipment (even if we add a joystick or a gamepad) easily fits into a standard backpack and the Hector GV(s) can be carried by hand. To start a mission, the robots and the laptop have to be switched on, and the operator can connect to the robots via Wireless LAN.

3 Communications

Our communication concept is based on two different channels. A common wireless network is used for high-bandwidth data like video images or map information. Currently we use a 2.4 GHz 802.11g/n network, but our hardware also allows 5 GHz or 802.11a/n operation if necessary. For data exchange with lower bandwidth demands the vehicle is additionally equipped with a 802.15.4 radio modem. This low-bandwidth link is used for telemetry and basic manual control of the vehicle and enables the operator to take over even when the onboard computer is no longer operational. The operator station is connected to a modified wireless access point which interfaces both networks, 802.11a/g and 802.15.4.

communication channels used

Technology	Power	Bandwith (nominal)
2.4 GHz – 802.11g channel 1-13	32 mW	54 MBit/s
5.0 GHz – 802.11a channel 36-54	32 mW	54 MBit/s
2.4 GHz – 802.15.4 channel 11-26	100 mW EIRP	115 kBit/s

4 Control Method and Human-Robot Interface

We focus more on autonomy than on mobility and manual control. In the ideal case the operator only has to monitor what the robot is doing and confirm the victim information the robot provides. In case the need arises, other control options are also available. Semi-autonomous operation can be performed by giving high level goals, modifying plans, and transferring critical decisions form the robots to the supervisor. Full manual teleoperated control as well as remote control via joystick/gamepad or keyboard is also possible.

Monitoring

Having used a software system based on RoboFrame [2] in previous years, we have selected ROS as a new middleware. While ROS already provides tools for debugging specific algorithms, we will continue to use parts of our previously developed GUI system, so an integrated GUI with all mission-critical elements is available to the operator. The mission control dialog provides a 3D view of the vehicle and map as well as information on battery status, vehicle attitude, behavior decisions as well as a pop-up with camera images once a potential victim has been found by the UGV.

For the supervision of more than one robot, a new communication concept has been developed, that is based on discrete events, rather than continuous data streams [9]. It provides the supervisor with information that is relevant for obtaining a situation overview, which is sufficient to recognize if the robots need any human support and for defining high-level goals. For direct teleoperation of a single robot, the operator has to switch to the more detailed interface.

Adjustable Autonomy

With the event-based communication described in the previous paragraph, the robots are enabled to transfer important decisions as queries to the supervisor. In full autonomy mode, this is only applied for confirming victims, however, the level of autonomy can be adjusted by transferring more decisions to the supervisor.

Furthermore, a method has been developed that allows the supervisor to modify the allocation of tasks to robots. For example, the supervisor can define experts for specific task types, or can forbid some robots to execute specific tasks. For each subtask, several solutions with varying level of autonomy can be chosen. As an example, consider the task of driving to a certain position. This can be either accomplished autonomously (by choosing the waypoint autonomously or by a klick on the map), or by following a predefined path (draw the path in the map), or by full remote control (use a joystick). The idea of adjustable autonomy can also be applied to other tasks like object detection or mapping.

5 Map Generation/Printing

The Simultaneous Localization And Mapping (SLAM) problem is solved by using a 2D grid map representation that gets updated using a scan matching approach. Our approach has low runtime requirements and can run with an update rate of 40Hz while requiring less than 15% CPU time on our Core 2 Duo setup, freeing resources for other computation. While capable of incorporating odometry, the currently used configuration works without odometry, as the scanmatching approach is very robust. The input used for solving the SLAM problem are laser scans and the robot state as estimated by the navigation filter (cf. section 6). Data provided by the navigation filter is used for transformation of laser scans to take into account the attitude of the laser scanner and vehicle during acquisition of scans. Figure 5 shows two maps learned using the Hector SLAM system. A video is available online [10].

To enable autonomous cooperative deployment of multiple robots on missions, a feature based map merging system is currently in development. Each robot detects SURF features [11] on the estimated map and these are exchanged among teammate robots. A registration approach is then used to arrive at a common coordinate frame for all robots.

To better negotiate the increasingly rough terrain in the rescue arena, we use a RGB-D camera mounted on the pan/tilt unit of the robot to acquire point clouds, build a 2.5D height map and classify the terrain into passable and impassable grid cells. Our software makes use of the Point Cloud Library (PCL) which is available as a ROS package.

The map can be manually or automatically annotated with information about hazmat symbols and victims. It can be converted and saved in the GeoTIFF format.

Maps learned with the Hector SLAM system: (a): GeoTIFF map of the RoboCup 2010 Rescue Arena. The robot was carried through the arena by hand for this dataset, thus no odometry was available. (b): Larger scale mapping of a university floor with loop closing.

Wheel Encoders: To measure the translational and rotational speed of the vehicle, all four wheels are equipped with incremental optical encoders. This odometry data is used especially for indoor navigation, but due to the inaccuracy additional feedback from other sensors is needed.

Laser Scanners: The vehicle is equipped with two laser scanners: A tiltable Hokuyo URG04-LX scanner is mounted in the front of the vehicle. It is mainly intended for scanning the ground in front of the vehicle. The second laser scanner, a Hokuyo UTM30-LX, is mounted on a roll/tilt unit at the front of the control box and is mainly used for 2D mapping. Both scanners can be stabilized to stay close to their intended scan plane regardless of vehicle attitude.

RGB-D Camera: We use a RGB-D camera for generating point clouds of the environment and to distinguish passable from impassable terrain. This camera is mounted on the pan/tilt unit that is also used for the thermal camera. We currently use the Microsoft Kinect sensor, but might exchange this for a smaller solution like the PrimeSense SDK 5.0 sensor once it is available.

Ultrasound Range Finders: Additionally to the laser scanners, a set of ultrasound range finders mounted at the back of the vehicle enables autonomous reactive collision avoidance when moving backwards, as the scanners only cover 270 degrees field of view.

Inertial Measurement Unit: To measure the attitude of the vehicle, it is equipped with a 6DoF inertial sensor ADIS16350 by Analog Devices which measures accelerations and angular rates.

Navigation filter: All sensor information is fused to get an overall estimate of position, velocity and attitude of the vehicle by using an extended Kalman filter. Although Kalman filtering is a common and simple approach for robot navigation problems, it suffers amongst others from the resulting unimodal representation of the belief state. On the other side, the feedback from map-based localization, as described in section 5, can lead to ambiguities which contradict the Gaussian assumption. Our approach is to combine these two sources of information in a loosely-coupled way in order to gain a robust navigation solution. The attitude estimate of the navigation filter is used to stabilize the laser scanners and camera system.

7 Sensors for Victim Identification

Finding human victims in the difficult conditions of unstructured post-disaster environments is one of the main goals of RoboCup Rescue. Significant progress in visual object recognition and scene understanding allows us to apply state of the art computer vision methods to tackle this problem. A comprehensive overview of the system we developed for semantic mapping using heterogenous sensors such as thermal and visual cameras can be found in [12].

Vision-Based Recognition of Victims and Hazmat Symbols: The recognition of the objects is performed by using a combination of visual cues based on the gradients of image intensity. Such cues can be efficiently captured by a descriptor based on the histograms of oriented gradients (HOG, see Fig. 3 for illustration). First, the gradient magnitude and orientation are computed densely in the image. The local distributions of the gradient orientation are then captured by the histogram. Such histograms are then grouped with their neighbors and jointly normalized. The normalization and local pooling of gradient information significantly improves the stability of the description to viewpoint changes, noise and changes in illumination.

It has been demonstrated that visual information represented in this way combined with powerful machine learning techniques can be successfully applied to recognition of people in realistic conditions [13]. While showing good performance this approach also requires significant processing power. The on-board computer (Fig. 4) with an nVidia graphics card allows real-time feature computation and recognition with an implementation based on [14].

We use the recognition system for detection of hazmat symbols at the victim sites (Fig. 4). The same system, but trained on the images of human body parts, is used to recognize victims parts.

In further work relevant to the USAR scenario [15], we improved part-based people detection algorithms for detecting people in arbitrary poses with partial occlusion by projecting the images to the ground plane, adding a scale prior, and combining the two best-performing algorithms. This leads to an equal error rate (EER) of 66%, compared to EER of 21.9% of the upper-body HOG detector. Some examples for both detectors can be seen in Fig. 5. However, because the objects we want to detect in the RoboCup scenario are not articulated humans, but rather rigid objects like baby dolls and hazmat signs, the HOG detector is sufficient, while requiring less computational power.

Multi-Cue Victim Detection: In addition to visual victim detection we will use a thermal camera as our secondary sensor. Thermal images often contain not only victims but also other warm objects, such as radiators or fire, so that thermal and visual recognition systems will deliver complementary information. This complementary information can be used to increase the reliability of our framework since victim hypotheses in thermal images can be verified with our visual inference method and vice versa. With this sensor fusion scheme many false alarms retrieved by single sensor systems can be avoided.

Original Image (left) and histogram of oriented gradients (right).

Our mobile computing platform with CUDA capable GPU (left) and uEye camera (middle), and a picture taken by the camera at RoboCup German Open 2009 in Hannover (right).

Several examples of detections at EER obtained with a HOG detector [13] (1st row), and the combined detector augmented with scale prior [15] (2nd row). True positive detections are plotted with yellow and false positives with red color.

8 Robot Locomotion

Our vehicle is based on a Kyosho Twin Force RC model with a powerful and fast drive train. For indoor navigation we modified the drive train, the steering and the suspension because of the much higher weight.

4-wheel-drive

The 4-wheel-drive of the vehicle has one differential gear per axis and no middle differential gear. This ensures that the vehicle is able to move when only a portion of wheels have ground contact. To reduce the maximum speed for indoor operation and to increase the torque we added a 1:5 gear.

4-wheel-steering

The front and rear wheels can be controlled independently, providing three advantages over normal 2-wheel-steering: (1) a smaller minimum turn radius (half of 2-wheel-steering), (2) the possibility that the rear wheels use the same trajectory as the front wheels (if both steering angles are the same) (3) the possibility to move sidewards (up to 35 degrees to the longitudinal axis of the vehicle).

Normally the rear wheels are set to the same steering angle as the front wheels, so that the resulting trajectories are identical and the risk of obstacle contact is reduced. With this vehicle we have a very flexible, mobile and powerful platform which additionally has the advantage of providing precise odometry information.

9 Other Mechanisms

(empty)

9.1 Established Technologies from RoboCup Experience

From 2001 till 2008 the Darmstadt Dribbling Dackels participated in the 4-legged soccer league as a part of the German Team and won the world championship in 2004, 2005 and 2008. Since 2004 the Darmstadt Dribblers participate successfully in the humanoid kid-size league and won the world championship in 2009 and 2010, and the award for the best humanoid robot in 2009. Although Search and Rescue is a totally different application than soccer, the team Hector Darmstadt can make use of the experiences from the soccer teams and many tools that were developed in these teams can also be applied for Search and Rescue. RoboFrame: Although we are in the process of migrating more and more modules to ROS, the search and rescue specific parts of our software is mainly based on RoboFrame [2]. This framework supports teams of heterogeneous autonomous lightweight robots. RoboFrame supports modular software development and takes care of the communication between software modules running on the robots and with our graphical user interface.

XABSL: The high-level behavior is described as a hierarchical state machine with the Extensible Agent Behavior Specification Language XABSL [16]. This allows to easily extend the behavior and to reuse existing parts in different contexts. XABSL was originally developed for the behavior of soccer robots, but it was also applied to team cooperation of heterogeneous robots [17].

MuRoSimF: The Multi-Robot-Simulation-Framework [18] provides components for the simulation of a robot's motion and sensing capabilities with different levels of detail. MuRoSimF allows to test each component of the software separately by replacing all other part by ground truth data. After component testing, before using the real hardware, the whole system can be tested in a MuRoSimF based simulation.

9.2 Hardware Modularity

The complete hardware structure of our vehicle is shown in Fig.6. The intrinsic sensors and actuators are connected to an interface board which communicates with a PC/104 computer, which enables 6DoF navigation and allows basic autonomous driving. Most of the extrinsic sensors are connected to a separate on-board computer which is equipped with a state-of-the-art Core 2 Duo mobile CPU and a high-performance GPU for parallel computing. This "vision box" fulfills the more demanding tasks of mapping and visual detection of victims and hazmat symbols.

The separation of both components, even on hardware layer, simplifies independent testing and offers a high degree of flexibility. The vision computer can easily be mounted on other robots or used as a separate instrument for the evaluation of computer vision algorithms. The robot itself is used in various outdoor scenarios as fast and lightweight research platform.

10 Team Training for Operation (Human Factors)

The mission control dialog provides all crucial high level information about the ongoing mission to the operator, so unmanned vehicles can be supervised and controlled without detailed knowledge about their specific capabilities like kinematics and dynamics. UGVs classify terrain into passable and impassable sections, so they generally do not need external supervision for exploring the environment. High level control of multiple UGVs is thus possible without expert knowledge about the vehicles. However, depending on the situation, the autonomy level might has to be lowered, in which case a operator has more direct control of vehicles and thus has to have more detailed knowledge about them. In the RoboCup Rescue scenario, we use only one operator, as the number of robots employed simultaneously is small. For other scenarios, different operators can be responsible for different autonomy levels and tasks.

We train operators in using the mission control interface and in teleoperation of robots. As mentioned before, the focus on our research lies in autonomy, so training in teleoperation is not as comprehensive as for many teams focusing on teleoperation.

11 Possibility for Practical Application to Real Disaster Site

The Hector GV is a fast vehicle that allows for precise and versatile locomotion. The low weight is a big advantage for fast and flexible setup of the whole system. The most critical points are movement in very rough terrain and sensitivity against some basic environmental factors like humidity.

The strength of our approach is the elaborate reusable software, which is a reliable base for developing and extending our system. For practical application to real disaster sites we have to improve abilities in (partial) autonomy and plan to combine the system with other existing systems like an UAV (Quadrotor) and (mobile) sensor nodes. We hope to be able to give useful, flexible assistance to operators in managing disaster scenario within a few years.

12 System Cost