View Original PDF

O.I.T-Trial 2017 Team Description Paper

Kenzaburo Miyawaki, Mutsuo Sano, Yuki Inoue, Yutaka Hiroi, Satoshi Nishiguchi, Motoyuki Suzuki

Osaka Institute of Technology, Faculty of Information Science and Technology, 1-79-1, Kitayama, Hirakata-city, Japan; Osaka Institute of Technology, Faculty of Robotics and Design, Chaya-machi, Kita-ku, Osaka-city, Japan

https://sites.google.com/view/oit-trial

Abstract This paper describes the team O.I.T-Trial for the 2017 RoboCup @Home league to be held in Nagoya, Japan. We introduce two robots named ASAHI ('Rising Sun') and SUBARU ('Pleiades'). Our research goal is assistive robots for every day life in a home. To achieve this, we have developed various software and hardware modules through the RoboCup @Home competitions. We present the details of the modules and our approach to perform the complex RoboCup @Home tasks.

Introduction

O.I.T-Trial is the name of our Japanese team that participates in the competitive world of RoboCup. We have entered robots in the RoboCup Japan Open @Home league every year since 2011 with consistent success (1st place in RoboCup Japan Open @Home league 2013, 2015, 2016. 2nd place in RoboCup Japan Open @Home league 2011, 2012, 2014). Team members belong to the Faculty of Robotics and Design and the Faculty of Information Science and Technology at the Osaka Institute of Technology.

The robots, named ASAHI and SUBARU, have been designed to become assistants for daily home life. In particular, we focus on the people who have some kind of difficulties on the activities of daily life, such as elderly people, challenged people and their family.

We consider that the stable control and natural communication function are very important for the daily life supporting robots. To satisfy the points, our robots have various sensors to perceive the external environment, human state and objects. They also have a robotic arm which has wide range of movement for object manipulation. Additional to the hardware, we have developed many software modules for the object and person recognition, multi-modal interaction and actuator control. The modules are partially running upon the Open-RTM framework which helps the integration of distributed systems, so that easily replace and update a specific function.

This paper covers not only the details of the above-mentioned but also our scientific achievements.

The rest of this paper is organized as follows. The sections 2 and 3 show a brief overview of our robots and their hardware specifications respectively. In section 4 we explain our software modules and algorithms. Section 5 describes our spoken dialog system, and section 6 introduces our scientific contribution. Finally, we conclude this paper in Section 7.

O.I.T-Trial @Home Robots

Fig. 1 and Fig. 2 show our robots, ASAHI and SUBARU, for the RoboCup @Home league. They have slightly different exteriors, but internal hardware structure is almost the same. Differential drive vehicle, vertical movable robotic cylinder and single arm are base hardware components.

One significant difference is a robot avatar which is set on ASAHI's upper body (Fig. 1). The avatar has an experimental feature to achieve better humanrobot interaction. A research summary of this avatar will be explained in section 6.1.

Fig. 1. ASAHI
Fig. 1. ASAHI
Fig. 2. SUBARU
Fig. 2. SUBARU

Hardware

This section describes the hardware specifications.

3.1 Actuators and Controllers

The details of actuators and controllers are as follows;

  • Vehicle: Pioneer P3-DX. Max speed is 1.2m/s.
  • Torso: IAI Robotic cylinder which can move linearly along the vertical axis.
  • Arm: Our original 6-DOF arm is attached on the top of the torso. Fig. 3 and Fig. 4 show the joints' configuration and movable range of the arm. This arm is equipped with a 1-DOF gripper (Fig. 5). All joints of the arm and gripper are driven by Dynamixel actuators.
  • Controller: 2 Laptop-PCs connected with a Ethernet cable. One more laptop can be added as needed.
Fig. 3. The manipulator's DOF
Fig. 3. The manipulator's DOF
Fig. 4. Movable range of the manipulator
Fig. 4. Movable range of the manipulator

3.2 Sensors

The sensors of the robots are as follows;

– RGB-D camera: Xtion PRO LIVE is set on the root position of the arm. This sensor is used for object recognition. SUBARU has Kinect for Xbox One on its head for person recognition and tracking.

– Laser range finder: UTM30-LX is set on the bottom of their body. ASAHI has one more LRF on its torso for person tracking.

  • Microphone: A super-cardioid shotgun microphone is set on their head. In the case of SUBARU, an 8 channel microphone array is attachable instead of the shotgun microphone.
Fig. 5. Gripper of the robot arm
Fig. 5. Gripper of the robot arm

Software and Algorithms

We explain our software and algorithms in this section. Most part of the system is running on the Microsoft Windows OS. A part of spoken dialog system is running on the Ubuntu. Even if different OS based programs are used, we can easily integrate them over the Open-RTM framework.

4.1 Object Recognition

Object grasping is highly required for RoboCup@Home tasks, and it is essential for the daily life assistive robots too. Needless to say, a robot has to recognize the objects inside of its sight and identify the objects' positions for safe grasping. For this, our robots are equipped with RGB-D camera and can grab point cloud data.

After grabbing point cloud data, the robot extracts object regions for recognition. We use Point Cloud Library for extraction. The extraction process is as follows;

  • The points outside of the specified area are removed. The objects that are too far from the robot will be ignored.
  • Downsampling of the point cloud for faster extraction.
  • Planes are detected and eliminated by Random Sample Consensus (RANSAC) algorithm.
  • Euclidean distance based clustering is applied on the rest of the points.

By the above steps, we can get some object clusters as shown in Fig 6.

Next, recognition process follows the extraction. We calculate Bhattacharyya distance between H-S histograms of the extracted clusters and pre-built known objects' H-S color histograms for recognition. Each unknown cluster will be classified into the known object which has the nearest Bhattacharyya distance. Fig. 7 shows an example screen shot of the recognition program.

Fig. 6. Object cluster extraction
Fig. 6. Object cluster extraction
Fig. 7. Object recognition
Fig. 7. Object recognition

4.2 Mapping and Localization

Simultaneous Localization and Mapping (SLAM) is a fundamental function for mobile robots. We use the Mobile Robot Programming ToolKit (MRPT) and its Iterative Closest Point (ICP) module for mapping. The performance of ICP SLAM is enough to execute real-time mapping, which is essential for some tasks such as Restaurant. The accuracy is also precise as shown in Fig. 8, which was built while a navigation phase of the Restaurant task conducted at the 2015 RoboCup Japan Open @Home league.

Fig. 8. Example map built with the ICP module of MRPT
Fig. 8. Example map built with the ICP module of MRPT

Spoken Dialog System

We utilize open source speech recognition (Julius) and text to speech (Flite and Merlin) software for spoken dialog.

One advantage of our robot is a selectable microphone. SUBARU can select a super-cardioid shotgun microphone and an 8 channel microphone array (Fig. 9). The shotgun microphone can be available for conversations between single person and robot with simple software implementation. On the other hand, the 8 channel microphone array is used for sound source localization which is needed for some RoboCup@Home tasks, such as Restaurant. We use HARK, the open sourced sound localization software, with the microphone array.

Fig. 9. 8 channel microphone array
Fig. 9. 8 channel microphone array

Scientific Contribution

This section describes our scientific contributions to robotics research.

6.1 Robot Avatar

ASAHI has a small robot avatar on its upper body (Fig. 1). The avatar is used for smooth communication between a human individual and the robot. The avatar uses its arm to show it has recognized the human's pointing gesture.

Fig. 10 illustrates the avatar's behavior. When the human gestures to indicate a point on the floor, the robot should recognize this. Gesture feedback from the avatar allows the human to confirm that the robot has successfully recognized the pointing gesture. If the robot fails to recognize the gesture, the human should repeat it until the avatar confirms that the robot has done so.

Fig. 10. Gesture feedback using the robot avatar
Fig. 10. Gesture feedback using the robot avatar

6.2 Robust Person Following

Person following is an essential function for the RoboCup@Home tasks. We proposed an algorithm for tracking a target individual using a laser range finder in a real environment, such as a room or a corridor, where other people are also moving.

For the robust following, ASAHI has a LRF to detect a person's waist (Fig. 11). The detection algorithm using the LRF is as follows;

    1. Split objects and walls by comparing the distances. (segmentation)
    1. Discriminate objects from walls using the width of the region. (discrimination)
    1. Calculate the position of the object. (position calculation)
    1. Search for the nearest object from the detected objects, which becomes the target to follow.
Fig. 11. Installation height of LRF
Fig. 11. Installation height of LRF

Conclusion

In this paper, we described our team and robots. Our main purpose is realization of assistive robots for daily home life. The robots are equipped with various sensors to perceive real world information and we have developed robust recognition systems and control software for the purpose.

The performance of our robots is well tested through many competitions of RoboCup Japan Open @Home league. Therefore, the robot will show good performance in the 2017 RoboCup World Competition @Home league. We continue to improve the robots for the 2017 RoboCup@Home league in Nagoya.

References

  1. Kenzaburo Miyawaki, Satoshi Nishiguchi, Motoyuki Suzuki, Yutaka Hiroi, Yuki Inoue, and Mutsuo Sano. Robocup japan open @home league : Assistive robot for daily life at home (in japanese). Systems, Control and Information, 60(2):73–78, 2016.
  2. Radu Bogdan Rusu and Steve Cousins. 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13 2011.
  3. Mobile robot programming toolkit. http://www.mrpt.org/. Accessed: 2017-02-20.
  4. Paul J. Besl and Neil D. McKay. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell., 14(2):239–256, February 1992.
  5. Yutaka Hiroi and Akinori Ito. Influence of the height of a robot on comfortableness of verbal interaction. IAENG International Journal of Computer Science, 43(4):447–455, 2016.
  6. Keita Futagami, Yutaka Hiroi, Hisanori Kuroda, Naoto Suzuki, and Akinori Ito. Proposal of a method to fetch an object on the floor using pointing gesture and spoken dialog (in japanese). In Proc. of The Robotics and Mechatronics Conference 2014, pages 2A1–F07, 2014.
  7. Yutaka Hiroi and Akinori Ito. Asahi: Ok for failure -a robot for supporting daily life, equipped with a robot avatar-. In Proc. of 8th ACM/IEEE International Conference on Human-Robot Interaction, pages 141–142, March 2013.
  8. Yutaka Hiroi, Shohei Matsunaka, and Akinori Ito. A mobile robot system with semiautonomous navigation using simple and robust person following behavior. Journal of Man, Machine and Technology, 1(1):44–62, 2012.
  9. Kohei Morishita, Yutaka Hiroi, and Akinori Ito. A crowd avoidance method using circular avoidance path for robust person following. Journal of Robotics, 2017, 2017. Article ID:3148202.