RoboCupRescue - Robot Rescue League Team Jacobs University Bremen, Germany

Soren Schwertfeger, Jann Poppinga, Kaustubh Pathak, Heiko Bulow, Narunas Vaskevicius, Andreas Birk

Jacobs University Bremen Campus Ring 1 28759 Bremen, Germany

http://robotics.jacobs-university.de/

Abstract The 2009 Jacobs University Bremen rescue robot team is described in this paper. The team is active in RoboCup rescue since 2001. The team takes an integrated approach to rescue robots, i.e., developing the systems from the mechatronics to the high-level functionalities for intelligent autonomous behavior, incorporating simulation in USARsim. The main extension of the contributions of the team in this year is work on 3D Mapping based on real-time registration of large surface patches.

Introduction

Jacobs University robotics group has been working in the domain of rescue robots since 2001. The team has already participated in the real robot rescue league at RoboCup 2002 in Fukuoka (4th place), RoboCup 2003 in Padua (4th place), the US Open 2004 in New Orleans (2nd place), RoboCup 2004 in Lisbon, RoboCup 2005 in Osaka, the Dutch Open 2006 in Eindhoven (1st place), the US Open 2006 in Atlanta (1st place), the RoboCup world championship 2006 in Bremen (best European), the German Open 2007 (1st place), RoboCup 2007 in Atlanta (best European), RoboCup German Open (1st place), and the RoboCup in Suzhou, China.

The team works toward autonomous intelligent systems that are fieldable. Therefore, an integrated approach is taken, i.e., the systems are developed at Jacobs University from the mechatronics level to the high-level software. The newest robot design is the the so-called "Rugbot" type (from "rugged robot", Fig. 1), which reflects this philosophy with its locomotion capabilities while having a significant potential for onboard intelligence. At the RoboCup 2005 competition in Osaka, the Jacobs University team was the only participant that managed to run with a single type of robot in the locomotion as well as in the autonomy challenge. Building on this, the team has since been the only one to do well both in autonomy and in locomotion.

The Robot Hardware

The "Rugbot" type of robot (Fig. 1) is the main type of robot used by Jacobs University robotics since several years. The robot is a complete in-house development designed especially for rescue applications. The implementation is based on the CubeSystem, a collection of hardware and software components for fast robot prototyping. The CubeSystem consists of the RoboCube controller hardware, a special operating system called CubeOS and libraries for common robotics tasks.

The Rugbots are tracked vehicles. They are lightweight (about 35 kg, with a newer generation weighing about 27 kg) and have a small footprint (approximately 50 cm x 50 cm). They are very agile and fast in unstructured environments and they also perform well on open terrain. A special feature of rugbot is an active flipper mechanism that allows to negotiate rubble piles and stairs. Rugbots have significant computation power in form of an onboard PC and they can be equipped with a large variety of sensors.

Control Software

The software architecture on the robots and the operators is designed to support the whole range from teleoperation to full autonomy. The core components of the software are used on the real robots as well as on the virtual robots of the Jacobs US-ARsim team. These methods and algorithms are bundled in a framework called Jacobs Intelligent Robotics Library (JIRlib). The JIRlib provides sophisticated, multithreaded C++ libraries for tasks like sensor-access, recoding and replaying of sensor data, mapping algorithms, autonomy, user interface, and networking.

Human-Robot Interface

Using the JIRlib a single operator can control all robots that are active in the field in parallel. The robots are semi-autonomous in the default mode. The operator is assisted by various autonomous functionality, especially map-building and identification of victims. The robots can also be run in full autonomous mode as demonstrated at RoboCup 2006 in the autonomy challenge.

For the human-robot interface work has been done to enable a single first responder to operate a whole set of robots. For this challenge, it is important to preprocess and streamline the immense data flow from the robots and to assist the operator as much as possible in the processes of controlling the robots. Therefore, an adaptive graphical interface was investigated that supports adjustable autonomy of rescue robots. In addition to providing standard interfaces in a flexible and dynamic way, it allows special features like a 3D display and change of viewpoints of 2D map data. In addition to this experimental interface, a standard GUI is used for the straightforward control of the robot (Fig. 2) which allows for most sensor data to be displayed to the operator along with a map.

Autonomy

The autonomy uses a behavior based approach which is guided by a path-planning algorithm. The autonomy thread runs in a continuous loop where at each sample time instant within this loop, all the sensors threads are sampled and their data is analyzed to fill in a situation object. This situation object has boolean flags for flagging various runtime conditions like nearby obstacles, dangerous pitch or roll, whether the robot is stuck or near a fall, whether any of the automatic victim detection algorithms found a victim, etc. Each such flag has its own statically assigned priority. Each flag is associated with an overall runtime severity value. Furthermore, this severity value is also provided directionally, e.g. if an obstacle is flagged, one can check all the different directions in which an obstacle has been detected, and what the relative severities are.

Depending on the situation object a appropriate behavior is chosen. A behavior is a complex sequence of motions executed by the robot in response to the situation. Behaviors can be aggregated together to form a new behavior. We distinguish between a behavior and a primitive motion like in-place rotation or pure translation. There is always an active (current) behavior which handles the situation, i.e., computes motion and actuator commands for the robot.

A new behavior is selected at each time instant based on the robot's perception. If a behavior has not finished its complete sequence of motions, it can ask the autonomy thread to consider running it in the next sample instant. This is, however, not guaranteed. This mechanism allows all behaviors to be interrupted in mid-run if a situation with a higher severity occurs which can best be handled by another behavior. Possible behaviors are Obstacle Avoidance, Largest Opening, Back Off, Victim Found or Motion Planner.

Map Generation and Path-Planning

Maps are completely autonomously generated on the Jacobs University robots. The user can choose online between different mapping algorithms:

• odometry-based probabilistic occupancy grids
• scan-matching with Grisetti's algorithm
• an algorithm based on forward sensor models
• an odometry-based 3D mapping algorithm

Each of the 2D maps can be used as a basis for path-planning for exploration in autonomous mode.

RoboCup related research on mapping of the group also includes results from successful mapping of large areas with multiple robots. The investigation of multi robot aspects is also done in research on exploration under the constraints of wireless networking. Further research is being done on the combination of metric and topological mapping for multiple robots.

Most recent work on mapping, which will also be the main contribution to our 2009 RoboCup activities, includes work on real-time 3D mapping using a fast registration of 3D scans through the extraction of large surface patches.

Onboard Sensors

A typical collection of the Rugbot onboard sensors is shown in Fig. 1.

Sensors for Navigation and Localization

The main range sensor used for navigation and map-generation is the low-cost Hokuyo laser range finder (LRF) URG04-LX. As shown in Fig. 1, some of the autonomous robots are equipped with an inclined LRF, which aids in avoidance of steep steps and ramps.

Optionally, the robots can also be equipped with a SICK LRF (S300 or LMS2xx) which provide a far greater range and intensity information on top of the range measurement. However, since they are quite bulky, they are only used in special scenarios, i.e. for an outdoor version of the robot that can withstand spray and a high-speed version that keeps in the center between two walls.

The main inclination and orientation sensor is a gyro based MTi from XSens which also contains a digital comapass. The robot odometry orientation is recalibrated using the gyro yaw angle at the rate of 10 Hz.

The motors of the robots are equipped with high resolution quadrature encoders from HP. The software modules of the CubeSystem not only use this data for control, but also for odometry and dead-reckoning to estimate the robot's pose. The according data contains relatively large errors and is subject to drift. But it nevertheless can be used together with other sensor data in estimation processes like Kalman filter. A further source of localization is the incorporation of the SLAM algorithm code from [GSB05] within our software framework. Refer to Fig. 4 for a comparison.

Sensors for Victim Identification

The main sensors for victim detection by a human operator via video are USB cameras from Philips. The cameras have a medium resolution of 640x480 pixels. The main advantage of these sensors is that they are low-cost. In the locomotion configuration, Rugbots are equipped with three to four of these cameras, in the configuration for autonomous operation, only one camera is used. Its image is used in detection algorithms.

A thermal camera is used in addition that not only provides data to the human operator but also to an autonomous vision module for victim detection (Fig. 5). This Flir A20 thermocam has a uncooled, high resolution Focal Plane Array (FPA). Its 160x120 elements provide temperature information in a range of -40°C to 120°C with 0.08°C resolution. The color to temperature map can be changed such that the related image highlights only spots with human body temperature. As shown in previous work, possible human victims are not only detected by temperature but also by shape.

Concretely, a novel approach to perception is used where a complete 3D scene model is learned on the fly to represent a 2D snapshot. An evolutionary algorithm (EA) generates pieces of 3D code. These are rendered and the resulting images are compared to the current camera picture via a special image similarity function. Based on the feedback of this fitness function, a crude but very fast online evolution generates an approximate 3D model of the environment where non-human objects are represented by boxes. 3D models of humans are available as code sniplets to the EA, which can use them to represent human shapes. The code sniplets for 3D boxes are mainly used by the EA to generate representations of occlusions. Successful experiments indicate that even humans in difficult poses can be recognized.

Like in previous years, CO₂ probes on the robots can be used to detect breathing victims. The Vaisala sensors of the previous years have been replaced by Sensair models which are several orders of magnitude less expensive. The main disadvantage of both these sensors is their long delay of up to 30 sec in the sensitivity.

Last but not least, a two-way sound connection between robots and operator station can be used by the human operator to identify, locate and console victims. Sound can also be used for victim detection in autonomous mode.

Another technique to detect victims is motion-detection. The software framework provides a basic movement detector which is activated periodically after stopping the robot. Two successive frames are taken from the USB camera and compared. If a certain amount of pixles change above a certain threshold, a movement is detected. Several movement detections in a row lead to the assumption that a moving victim is found. Aside from victim, stickers indicating hazardous materials ("Hazmats") can also be detected in the camera image by their characteristic shape.

3D Sensors

As shown in Fig. 1, the robot is equipped with two primary 3D sensors: a time-offlight CSEM Swiss-ranger, and a SR-3000 Stereo-on-Chip (STOC) Stereocam. Typical outputs from these sensors have been shown in Fig. 6. Our team has been sucessfully working on 3D data fusion and extraction of useful information.

This also includes work on terrain classficiation using 3D range data. The approach is to use a three-dimensional Hough transform. Scenes where one plane is dominant (floor, ramp) one bin is dominant in the discretized Hough space. According to the bin's parameters and the robot's properties, the scene can be classified as drivable or non-drivable. If there is not one dominant plane (rubble pile, random step field), an obstacle is reported.

Team Training for Operation (Human Factors)

Jacobs University has two on-campus rescue arenas (Fig. 7). The first one based on a high bay racking system is available since spring 2004. Its design allows to have a large floor-space and many different levels. The arena has a footprint of 5.60m by 4.70m and it is approximately 6m high. It has 3 main floors and several intermediate floors, which are interconnected. The second more recent arena is a reconstruction of a typical scenario found in RoboCup Rescue League on an area of 27m by 25m. It features several random-step-fields, two levels connected by stairs, a bridge, several ramps and perception challenges, as well as simulated victims. A separate course to test autonomy is also provided. The arena is available to external teams for training, benchmarking, etc.

As of spring '08, validation of a Rugbot model for the Urban Search and Rescue Simulator (USARsim) is expected. This means that the same robot can be used in reality and in simulation. What is more, since the control software for physical and virtual robots use the same API, components like mapping or planning algorithms can be tested in the simulator before applying them in the real world.

System Cost

The costs for each bare robot with control and locomotion system plus on-board PC is in the order of 8,000 Euro. The most expensive single sensor is the Flir A20 thermo cam with 16,000 Euro. The standard sensor load of each robot costs in the order of 4,000 Euro. Some detailed information on components and suppliers is located at http://robotics.jacobs-university.de/

Acknowledgments

The authors gratefully acknowledge the financial support of Deutsche Forschungsgemeinschaft (DFG) for their research.

Please note the name-change of our institution. The Swiss Jacobs Foundation invests 200 Million Euro in International University Bremen (IUB) over a five-year period starting from 2007. To date this is the largest donation ever given in Europe by a private foundation to a science institution. In appreciation of the benefactors and to further promote the university's unique profile in higher education and research, the boards of IUB have decided to change the university's name to Jacobs University Bremen (Jacobs). Hence the two different names and abbreviations for the same institution may be found in this paper, especially in the references to previously published material.

References

1. [BC06a] Andreas Birk and Stefano Carpin. Merging occupancy grid maps from multiple robots. IEEE Proceedings, special issue on Multi-Robot Systems, 94(7):1384–1397, 2006.
2. [BC06b] Andreas Birk and Stefano Carpin. Rescue robotics a crucial milestone on the road to autonomous systems. Advanced Robotics Journal, 20(5):595–695, 2006.
3. [BCC⁺06] Andreas Birk, Stefano Carpin, Winai Chonnaparamutt, Viktoras Jucikas, Hamed Bastani, Ivan Delchev, Ivan Krivulev, Seongchu Lee, Stefan Markov, and Andreas Pfeil. The IUB 2005 rescue robot team. In Itsuki Noda, Adam Jacoff, Ansgar Bredenfeld, and Yasutake Takahashi, editors, RoboCup 2005: Robot Soccer World Cup IX, Lecture Notes in Artificial Intelligence (LNAI). Springer, 2006.
4. [BCK04] Andreas Birk, Stefano Carpin, and Holger Kenn. The IUB 2003 rescue robot team. In D. Polani, B. Browning, A. Bonarini, and K. Yoshida, editors, RoboCup 2003: Robot Soccer World Cup VII, volume 3020 of Lecture Notes in Artificial Intelligence (LNAI). Springer, 2004.
5. [Bir04] Andreas Birk. Fast robot prototyping with the cubesystem. In Proceedings of the International Conference on Robotics and Automation. IEEE Press, 2004.
6. [Bir05] Andreas Birk. The IUB 2004 rescue robot team. In Daniele Nardi, Martin Riedmiller, and Claude Sammut, editors, RoboCup 2004: Robot Soccer World Cup VIII, volume 3276 of Lecture Notes in Artificial Intelligence (LNAI). Springer, 2005.
7. [BK03] Andreas Birk and Holger Kenn. A control architecture for a rescue robot ensuring safe semi-autonomous operation. In Gal Kaminka, Pedro U. Lima, and Raul Rojas, editors, RoboCup-02: Robot Soccer World Cup VI, volume 2752 of LNAI, pages 254–262. Springer, 2003.
8. [BKR⁺02] Andreas Birk, Holger Kenn, Martijn Rooker, Agrawal Akhil, Balan Horia Vlad, Burger Nina, Burger-Scheidlin Christoph, Devanathan Vinod, Erhan Dumitru, Hepes Ioan, Jain Aakash, Jain Premvir, Liebald Benjamin, and Luksys Ge. The IUB 2002 rescue robot team. In Gal Kaminka, Pedro U. Lima, and Raul Rojas, editors, RoboCup-02: Robot Soccer World Cup VI, LNAI. Springer, 2002.
9. [BKS02] Andreas Birk, Holger Kenn, and Luc Steels. Programming with behavior processes. International Journal of Robotics and Autonomous Systems, 39:115–127, 2002.
10. [BKW00] Andreas Birk, Holger Kenn, and Thomas Walle. On-board control in the RoboCup small robots league. Advanced Robotics Journal, 14(1):27–36, 2000.
11. [BMDP06] Andreas Birk, Stefan Markov, Ivan Delchev, and Kaustubh Pathak. Autonomous rescue operations on the iub rugbot. In IEEE International Workshop on Safety, Security, and Rescue Robotics (SSRR). IEEE Press, 2006.
12. [BP06] Andreas Birk and Max Pfingsthorn. A hmi supporting adjustable autonomy of rescue robots. In Itsuki Noda, Adam Jacoff, Ansgar Bredenfeld, and Yasutake Takahashi, editors, RoboCup 2005: Robot Soccer World Cup IX, volume 4020 of Lecture Notes in Artificial Intelligence (LNAI), pages 255–266. Springer, 2006.
13. [BPC⁺07] Andreas Birk, Kausthub Pathak, Winai Chonnaparamutt, Soeren Schwertfeger, Ivan Delchev, Stefan Markov, and Ravi Rathnam. The IUB 2006 rescue robot team. In G. Lakemeyer, E. Sklar, D. Sorrenti, and T. Takahashi, editors, RoboCup 2006: Robot Soccer World Cup X, volume 4434 of Lecture Notes in Artificial Intelligence (LNAI). Springer, 2007.
14. [BPSC06] Andreas Birk, Kausthub Pathak, Soeren Schwertfeger, and Winai Chonnaparamutt. The iub rugbot: an intelligent, rugged mobile robot for search and rescue operations. In IEEE International Workshop on Safety, Security, and Rescue Robotics (SSRR). IEEE Press, 2006.
15. [BSP09] Andreas Birk, Sren Schwertfeger, and Kaustubh Pathak. A networking framework for teleoperation in safety, security, and rescue robotics (ssrr). IEEE Wireless Communications, Special Issue on Wireless Communications in Networked Robotics, 2009.
16. [CBJ05] Stefano Carpin, Andreas Birk, and Victoras Jucikas. On map merging. International Journal of Robotics and Autonomous Systems, 53:1–14, 2005.
17. [CLW⁺06] Stefano Carpin, Michael Lewis, Jijun Wang, Stephen Balakirsky, and Chris Scrapper. Bridging the gap between simulation and reality in urban search and rescue. In RoboCup 2006: Robot Soccer World Cup X, LNAI, pages 1–12. Springer, 2006.
18. [CSE06] CSEM. The SwissRanger, Manual V1.02. CSEM SA, 8048 Zurich, Switzerland, 2006.
19. [dar98] Distributed Autonomous Robotic Systems 3. Springer, 1998.
20. [GSB05] Giorgio Grisetti, Cyrill Stachniss, and Wolfram Burgard. Improving grid-based slam with rao-blackwellized particle filters by adaptive proposals and selective resampling. In Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, 2005.
21. [htt] http://www.robocuprescue.org/wiki/index.php?title=Virtualrobots. Robocup virtual robots competition.
22. [Ken00] Holger Kenn. CubeOS, The Manual. Vrije Universiteit Brussel, AI-Laboratory, 2000.
23. [MB07] Stefan Markov and Andreas Birk. Autonomous victim detection in rescue operations by modelbased reproduction of 3d scenes from 2d thermal images. In Joachim Hertzberg, Michael Beetz, and Roman Englert, editors, KI 2007: Advances in Artificial Intelligence, volume 4667 of Lecture Notes in Artificial Intelligence (LNAI). Springer, 2007.
24. [ME85] H. Moravec and A. Elfes. High resolution maps from wide angle sonar. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 116–121. 1985.
25. [PBP08a] Kaustubh Pathak, Andreas Birk, and Jann Poppinga. Subpixel depth accuracy with a time of flight sensor using multimodal gaussian analysis. In International Conference on Intelligent Robots and Systems (IROS), Nice, France, 2008. IEEE Press.
26. [PBP08b] Jann Poppinga, Andreas Birk, and Kaustubh Pathak. Hough based terrain classification for realtime detection of drivable ground. Journal of Field Robotics, 25(1-2):67–88, 2008.
27. [PBss] Jann Poppinga and Andreas Birk. A novel approach to wrap around error correction for a time-of-flight 3d camera. In Luca Iocchi, Hitoshi Matsubara, Alfredo Weitzen-feld, and Changjiu Zhou, editors, RoboCup 2008: Robot WorldCup XII, Lecture Notes in Artificial Intelligence (LNAI). Springer, in press.
28. [PBSP07] Kaustubh Pathak, Andreas Birk, Soeren Schwertfeger, and Jann Poppinga. 3d forward sensor modeling and application to occupancy grid based sensor fusion. In International Conference on Intelligent Robots and Systems (IROS), San Diego, USA, 2007. IEEE Press.
29. [PSV07] Max Pfingsthorn, Bayu Slamet, and Arnoud Visser. A scalable hybrid multi-robot slam method for highly detailed maps. In RoboCup 2007: Proceedings of the International Symposium, LNAI. Springer, 2007.
30. [PVBP08] Jann Poppinga, Narunas Vaskevicius, Andreas Birk, and Kaustubh Pathak. Fast plane detection and polygonalization in noisy 3d range images. In International Conference on Intelligent Robots and Systems (IROS), Nice, France, 2008. IEEE Press.
31. [RB05] Martijn Rooker and Andreas Birk. Combining exploration and ad-hoc networking in robocup rescue. In Daniele Nardi, Martin Riedmller, and Claude Sammut, editors, RoboCup 2004: Robot Soccer World Cup VIII, volume 3276 of Lecture Notes in Artificial Intelligence (LNAI), pages pp.236–246. Springer, 2005.
32. [RB06] Martijn Rooker and Andreas Birk. Communicative exploration with robot packs. In Itsuki Noda, Adam Jacoff, Ansgar Bredenfeld, and Yasutake Takahashi, editors, RoboCup 2005: Robot Soccer World Cup IX, volume 4020 of Lecture Notes in Artificial Intelligence (LNAI), pages 267–278. Springer, 2006.
33. [USA06] USARsim. Unified system for automation and robotics simulator (usarsim), 2006.
34. [VBPP07] Narunas Vaskevicius, Andreas Birk, Kaustubh Pathak, and Jann Poppinga. Fast detection of polygons in 3d point clouds from noise-prone range sensors. In International Workshop on Safety, Security, and Rescue Robotics (SSRR). IEEE Press, 2007.
35. [VD06] Videre-Design. Stereo-on-a-chip (STOC) stereo head, user manual 1.1. Videre Design, February 2006.