RoboCup Rescue 2022 Team Description Paper iRAP ROBOT

Peeyaphoom Thanawutthianan, Jirakarn Sukcharoen, Artit Narasetthakul, Chaiyapruk Laohapanich, Noppadol Pudchuen

King Mongkut's University of Technology North Bangkok

http://www.kmutnb.ac.th/en/ · https://robocup-rescue.github.io/team_description_papers/ · https://drive.google.com/drive/folders/1YOye4kkhI27qJ93c4SON6sUN3819or5t?usp=sharing

Abstract This paper describes the improved construction and operation of our robot, iRAP ROBOT, which has a long history participating in the World RoboCup Rescue Robot competitions in the name such as iRAP PRO, iRAP JUDY, iRAP FURIOUS, iRAP JUNIOR, iRAP ROBOT teams. During our participation in the World RoboCup Rescue Robot 2019 (Sydney, Australia), we had several issues that caused us bad score such as autonomous operation and 3-D map construction. Therefore, in the upcoming World RoboCup Rescue Robot 2022 in Bangkok, Thailand, we are aiming to tackle these two issues seriously. The detail will be described in this Team Description Paper. Our semi-autonomous robot's locomotion mechanism still use a caterpillar type with pieces of garden house as a traction track for caterpillar's chain. This structure can help moving up and down the inclined surface with ease. The ability to identify victims is based on several sensors such as carbon sensors, array temperature sensors, and cameras. Our robot can be programmed to move autonomously in the radio drop zone. Additionally, it can create an explored map automatically and can detect the Hazmat and the QR code.

I. INTRODUCTION

INVIGORATING ROBOT ACTIVITY PROJECT (iRAP) is the name of a group of students from King Mongkut's University of Technology North Bangkok, Thailand. This year, our robot is named "iRAP ROBOT". The team members are consisting of both the previous members who won the 1st place many times in the RoboCup Rescue Robot competition [1] and the next generation of enthusiastic students. For more than nine years, our team has designed and developed the rescue robots as the semi-autonomous robot shown in fig. 1. This robot can roam around on a rough terrain using the caterpillar module. Four cameras are installed on the robot to help identifying the victims. Moreover, the temperature camera and CO₂ sensor are attached to the end-effector of the robot arm. The laser range finder is used to create the map as well as marking the victims autonomously. These functions of our robot are built not only to tackle challenges of the World RoboCup Rescue competition, but they are created in hope of one day our robot can be deployed to help human in the real catastrophic events such as earthquakes.

A. Improvements over Previous Contributions

According to the RoboCup rescue rule of 2021, the robot needs to improve mechanical and electrical systems to serve the functioning of autonomous operations and to have less weight than the previous version. For this reason, the mechanical team has decided to redesign structure by replacing aluminium with lightweight composite material as shown in fig. 2. The base design has remained the same since 2013, but the robot's interior continues to be developed. During the competition, our team has to maintain the robot game by game. There is a tight time limit to fix the problems that can occur at a one-stop formula pit. We have designed our robot such that we can remove and reinstall parts of the robot as fast as possible. The robot is outstanding in the best-in-class mobility category. To perform with excellent robot mobility in the rock and sand field is always our challenge. In 2019, we installed the seal part to prevent sand from going inside the robot's body and this year we also improve some parts of robot to avoid the sand better than before. This is because sand can damage the mechanic and electronic systems and cause the whole system to shut down. However, the seal blocks the air ventilation and keeps the high temperature inside the robot. The high-temperature levels can cause errors resulting in an illegal operation or freezing. The severe problem occurred to our team in the final round of RoboCup 2017. The robot continued operating and got stuck; then, the operator tried to move the robot out using a boost-up drive function. At that time, the power was overloaded due to the accumulated heat. The robot caught fire. Since this unforgettable experience, we have taken good care of the power consumption, temperature monitoring, and cooling system. Another important module is the robot arm and the multi-functional gripper as shown in fig. 3.

A. Improvements over Previous Contributions (continued)

This traditional structure was designed since 2017. We have the reliable main structure of the robot arm. We strengthen the mechanism to prevent harm to the robot arm and the gripper from the dynamic load and shock load while the robot moves, as presented in fig. 4. Moreover, it needs to upgrade to have high mobility and good precision using kinematics model. The other functions is to design by solving problems that occurred from the last competition and to improve the ease of robot control and QR code detection. Our primary areas of focuses are exploring all areas, detecting all victims, motion detection, map merging, self-drive autonomy, and generating a map in 2-D and 3-D. The simulated situation included many rough surfaces, hard terrains, rolling floor, stairs, and inclined floors; therefore, the rescue robot should be fast, light-weight and robust enough to circulate and explore.

II. OVERVIEW SYSTEM

Our robots have developed mechanical parts, electronic circuits, and software for controlling the robot as shown in fig. 5. The best mobility awards in the World RoboCup Rescue competition can guarantee the capability of our robot. We have experimented with the robot motion to address the difficulties in the different terrains. Moreover, we have developed software that is parallel with the proper hardware. We try to start it up in a more advanced system. The details of our robot are as follows:

A. Hardware

The semi-autonomous robot weighs in at 77 kg, 1 m long, 60 cm wide, 60 cm high, and can reach up to 2 m when fully standing up by placing our flipper vertically to the ground. The semi-autonomous robot has four flippers for balance against all kinds of obstacles. It also can flip itself back. The flippers are modular in design, meaning that it can still run with just one, two, or three flippers, or even with no flipper at all. It has a 1.5 m detachable arm with 360-degree rotation and a hand that you can shake, with a servo motor to reach small spaces. It is equipped with four cameras: two at the base, one front camera, one back camera, and two on the arm that is the primary search cameras. It has three sensors: two in the front for CO₂ detection, and one laser scanner in the back to map out the surrounding within a 50 meters radius. It is equipped with a flashlight for navigation, a microphone, and a speaker, allowing the rescuers to communicate with the survivors. It took us many years to develop the semi-autonomous robot to achieve the highest robotic mobility. There are plenty of ups and downs, but we have never given up. We had to overcome several obstacles, including developing the right design, and then convince a sponsor to provide us with materials. We converted a readily available commercial converter belt to the continuous track for the robot. With this design, the robot can navigate through various terrain, including rocks, sand, debris, and other surfaces. The autonomous mode robot can move and explore the map automatically. It has four cameras to detect QR code, CO₂ sensor, and a thermal camera to detect the victims/C gap, as in fig 6. The autonomous robot partially succeeds in the previous competition. Refer to the Table I and the following as well as Table II in the Appendix B.

Locomotion

The team is looking forward to researching the better locomotion system. The locomotion of all semi autonomous robots made up of the conveyor belt system that the team examined from different surface characteristics of the terrain. Many parts of the robot have been improved in order to be tough, lightweight and easy for maintenance. Each drive system consists of two motors: 24V, 325 rpm DC with gear-boxes for movement on the left and right. The structure of the drive system is made of aluminum. Synthetic rubber is used to make the belt. The robots have a pair of flippers that can be rotated 360 degrees.

Power (Batteries)

The robot uses LiPO batteries as our main power source because they are light and have high energy. The electronics systems are low-level systems. The micro-controllers are used to interface with the motor-driven system and data acquisition.

Manipulation or directed perception

The semi-autonomous robot can extend its arm up to two meters to search for victims around the disaster area. The inspection arm consists of a temperature and CO₂ sensors. The robot arm can navigate itself by estimating the end-effector position in the cartesian coordinate system.

Sensors

For victim identification, we will analyze the information from many kinds of sensors located on the robot inspection arm. For the preliminary step, we will check the status of the victims through the CCTV camera and measure the victim's body temperature utilizing the sensor. In some circumstances, this victim temperature information will be incorporated with the data from the CO₂ sensor and the surrounding sound, which will be received via the microphone, to analyze the situation of the victim.

B. Software

Several kinds of sensors are installed on the robot to acquire the data for processing and creating an automatic 2-D map on the operator's computer monitor. The map is generated by using the distance of the robot movement from rotary encoders, the inclination of the robot, the direction of the robot sensed by the Inertia Measurement Unit, and the distance between the robot and obstacles from the laser range finder. However, when the robot moves on different kinds of surfaces, slipping is unavoidable. This slipping is a significant problem in terms of designing and constructing the robot. Therefore, the SLAM algorithm [2], [3] is utilized to help generate the map, along with the information from the rotary encoders, and estimates positions on the X-axis, Y-axis, and Z-axis from the camera with the LiDAR scan (refer to Table III in the Appendix B). Several kinds of sensors are installed on each robot to gain crucial information about the environment around the robot. A thermal camera and CO₂ detectors are mounted at the end-effector of the robot's manipulator to detect the heat signature and CO₂ level of the immediate environment. The position estimation for the robot is determined by fusing pulses from encoders, the inclination from IMU, and the direct distance from LiDAR sensors, which are then used to generate the 2-D Occupancy grid map of the environment by utilizing the SLAM library available on Robot Operating Systems (ROS). Moreover, to detect visual clues from the immediate environment (such as a hazmat sign, QR codes, or motion) the algorithm that auto-detects these clues was also implemented (refer to Table III in the Appendix B). Since 2019, we have been developing 3-D map for robot so it can navigate through the sophisticated working space and dynamic environment of independent experiment by feature extraction and plotting the environment with dense point cloud information from the RGB-D camera [4].

C. Communication

There are communication systems used between the operator and the robot. It uses a wire/wireless LAN based on the IEEE 802.11ac standard as the main communication system. That is, it controls robots, receives video streaming from cameras on robots, and checks sensor feedback to determine the status of the robots on a computer monitor as well as for automatic map generation.

D. Human-Robot Interface

The user interface is running on ROS to support the future autonomous operation. The operator can quickly get used to controlling the robot using a typical gaming joystick, two information displays, and a communication headset. The gaming joystick is used as input to control the robot, and can easily be configured according to different operator preferences. The display is used to visualize the quad-channel video feed from the robot, the robot's orientation, and essential information, as shown in fig 7. Visual clue detection (motion, hazmat, and QR codes) and CO₂ are shown in fig 8.

The main control is based on one CPU (32-bit microcontroller). Fig 5 shows the diagram of the control system, which has two main tasks, as follows:

• To receive the data for identifying the status of the robots as shown in fig 7 (Quad-video and sensors information) and create 2-D maps automatically as shown in fig 9. This information will be shown to the robot operator via a second computer monitor.
• Data to control the movement will be sent via signal to the drive control of the DC motor at various locations on the robots. The RS-232 communication system will be used for sending and receiving the data of the CPU. Therefore, there must be a serial server to convert the RS-232 system to the Ethernet system.

III. APPLICATION

[Application section introduction]

A. Set-up process

The speed of the set-up and process of each task is crucial. The team realizes that the faster setup process, the more time for other tasks. The team uses the rugged plastic case as the station. When needed, the operator station will be opened and the switch will be turned on. The operations can be started within one minute. Inside operator station, there is one monitor, a laptop, an access point, and a UPS as in fig 10. Once all the tasks are completed, the data logger and the generated map can be quickly reported. Less is more: The GUI and the status display software are being combined into one motor.

B. Experiments

We put the semi-autonomous robot to the test by participating in the World RoboCup Rescue Robot competition in 2018. During the competition, the objective is to find as many survivors as possible within a time limit. We were able to win those competitions due to the robot's unsurprising ability to navigate through various terrains, the camera, and the CO₂ sensors' capabilities to locate and identify whether survivors are alive. Moreover, seeing the generated path on the map, the robot can automatically generate a map and mark survivors' locations, and the rescuer can compare that to the blueprint of the building. We believe this feature is one of the key differences that put us in the best mobility category. In 2019, we got second and "best in class mobility" prize because we do not have the self-drive/autonomous mode. Then the target in this year that is self-drive standalone task in some mission of World RoboCup Rescue Robot competition in 2022 for improving and developing our knowledge, potentiality, and technology.

C. Hazmat Recognition

The robot has a camera on the manipulation arm to detect the hazmat tags. This year, our greatest success has been detecting the hazmat tags, using the machine learning method called YOLO [5], as shown in fig 11 (refer to Table III in the Appendix B). All 21 tags are used in the competition. Our model for recognizing the hazmat tags is shown in fig 12.

IV. CONCLUSION

The competition leads us to eagerly research the ways to rescue humans when the world needs it. After the competition, the team knew how to make the better robots. The team had learned and exchanged about new technologies with other countries' competitors and learned how to have a good teamwork.

APPENDIX A: TEAM MEMBERS AND THEIR CONTRIBUTIONS

The iRAP ROBOT has 17 members and 8 advisors. The names and responsibilities of each member are listed as follows:

Team Members and Responsibilities

APPENDIX B: LISTS

There are four main systems:

• The Hardware System in the Tables I
• The Hardware Components List in the Tables II
• The Software List in the Tables III
• The Operator Station in the Tables IV

A. Systems List

[Systems List introduction]

TABLE I: HARDWARE SYSTEM

TABLE II: HARDWARE COMPONENTS LIST

TABLE III: SOFTWARE LIST

TABLE IV: OPERATOR STATION

ACKNOWLEDGMENT

The authors would like to thank the King Mongkut's University of Technology North Bangkok for regularly supporting the iRAP Robot in participating the World Robocup Rescue Robot Competition.

References

[1] A. Phunopas, A. Blattler, and N. Pudchuen, "irap robot: World robocup rescue championship 2016 and best in class mobility award," Robot World Cup. Springer, Cham, 2016. [2] R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots. The MIT Press; second edition edition, 2011. [3] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics. The MIT Press, 2005. [4] N. Pudchuen and W. Jitviriya, "3-d mapping and localization using rgb-d camera," in First International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP), 2019. [5] J. Redmon and A. Farhadi, "Yolo9000: Better, faster, stronger," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525, 2017. [6] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 1, 2001, pp. I–511–I–518 vol.1. [7] R. Lienhart and J. Maydt, "An extended set of haar-like features for rapid object detection," in Image Processing. 2002. Proceedings. 2002 International Conference on, vol. 1, 2002, pp. I–900–I–903 vol.1.