KIT Happy Robot 2018 Team Description

Kosei Demura, Koyo Enomoto, Kazuki Nagashima, Takeaki Yamakawa, Ryo Iwasaki, Syuhei Mashimo

Kanazawa Institute of Technology, Department of Robotics 7-1 Ohgigaoka Nonoichi, Ishikawa / 921 – 8501, Japan

Abstract This paper describes our research interests and technical information of the KIT Happy Robot team, former Happy Mini, for the RoboCup @Home league of RoboCup 2018. The KIT Happy Robot team from the Kanazawa Institute of Technology has developed an autonomous domestic service robot for little child and elderly. This paper describes the basic architecture of the robot as well as present various algorithms and research contribution. Finally, this paper concludes and outlines our future research works.

Introduction

The KIT Happy Robot team, our former team name was happy mini, has been participating in the @Home league of the RoboCup Japan Open since 2012, and participated in the RoboCup world competition since 2015. Our team got the 9th in 2015, 8th in 2016, 9th in 2017, respectively. Our team is a joint team of the Yumekobo project and the Demura research laboratory at the Kanazawa Institute of Technology (KIT). The Yumekobo, the factory for dreams and ideas, is a unique educational system in KIT. In 1993, KIT established Yumekobo to encourage students to create things and make character building. The most important activity of Yumekobo is to support student projects called Yumekobo Projects whose purpose is to improve technology and teamwork.

The vision of the KIT Happy Robot team is "Making the World Happy by Making a Kawaii Robot". "Kawaii" is an adjective in Japanese. The meaning is cute, lovely or charming. We think that the kawaii robot can solve problems of an aging society in Japan and make people happy. Robots in the @Home league should be more kawaii.

Our robots as shown in Fig.1 is designed in the image of a little child with a lively yellow color. Our Robots are considered to be the first kawaii designed robots in the RoboCup @Home league. The software system is based on ROS, and the state of arts frame works such as Darknet, DIGITS, Hark, Kaldi and so on.

The rest of paper is as follows. Section 2 describes the architecture of our robots, Section 3 describes the research contribution. Finally, this paper concludes by exploring applications and denoting future work.

Architecture

Design Concept

Our robots have two design concepts Kawaii and Safety.

The first concept of our robots is Kawaii. Our robot is used for childcare and conversation partner for elderly, and also for persons feel lonely living alone. Kawaii robot can ease those people. Thus the exterior design is crucial, and a lovely and friendly exterior design is considered to be suitable for those people.

The second concept of our robots is Safety. Safety is the most important aspect of service robots for children and elderly. Our robots are lightweight less than 10[kg], minimal power, and the joint parts are covered to prevent fingers cannot be inserted as shown in Fig.2.

Hardware

The platform of our robot is the Kobuki base that is a low-cost mobile research base designed for education and research. The torso is the extensible, and the commercial electric extendable cane, the KODUECHAN (ITK Co.Ltd), is used. It is only 0.38[kg], rated for up to 100[kg], extends and retracts to 250[mm]. The Robot1 as shown in Fig.9 has the 4-DOFs (shoulder 1, elbow 1, wrist 1, hand 1) arm. The maximum load of this arm is 0.5[kg]. The Robot2 as shown in Fig.10 has the 5-DOFs (shoulder 1, elbow 1, wrist 2, hand 1) arm. The maximum load of this arm is 0.78[kg]. Each arm of two robots were fully developed by our team.

The hand is also designed to have a capability of grasping an object, from ground up to 110[mm] on the ground. Our robot has various sensors, such as the RGB-D sensor (Realsense, Intel), the LIDER (UTM-30LX, Hokuyo), and the 360 degree camera (4KVR360, Kodak).

Software

Speech Recognition & Speech Localization

We have developed a speech recognition an source localization system. The speech recognition system uses the Kaldi [1] gstream server. It is a real-time full-duplex speech recognition server, and uses a DNN-based model for English trained on the TEDLIUM speech corpus. The sound source localization system is implemented using the robot auditory library HARK [2]. The system takes multi-channel speech waveform data from the 8ch microphone array, calculates FFT and estimates the sound source direction by the MUSIC method.

Grasp an object on the ground (left) and high place (right)

Object Recognition & Manipulation

We use YOLO (You Only Look Once) [3] for object detection. YOLO is real-time object detection deep neural network. First, we cut out the image area of object using YOLO. Then object image that like cup-shaped or bottle-shaped image is classified each concrete object using CNN (LeNet).

The robot can recognize objects smaller supervised dataset using this method than using YOLO only. The result of object recognition is shown in Fig.4. Next, the robot removes the planer and outlier from point cloud data, and calculates the centroid of clustering from point cloud corresponded object image area. This calculation use Point Cloud Library (PCL) [4].

Finally, the robot calculates Inverse Kinematics from object center of gravity and grasp it.

The team developed an algorithm that can solve the significant problem of the waointbased navigation that is the positions of the waypoints can be located in unreachable areas due to errors in self-localization and the map [5].

Gmapping, Adaptive Monte Carlo Localization (AMCL) [6], and Dynamics Windows Approach (DWA) [7] algorithms are used for SLAM, localization, and collision avoidance, respectively.

The robot can avoid the small object using Ultrasonic Distance Sensor (PING, Parallax Inc.), and 3D object using Point cloud from RGB-D sensor (Realsense, Intel).

Research Contribution

The first author's laboratory has been developing an autonomous personal mobility called UNiMO AI based on commercially available crawler typed electric wheel chair. The conventional wheelchair is difficult to get over small difference (about 30 [mm]) in level, and it sometimes makes a falling accident. To prevent the accident, the capability of detecting the small difference is required. UNiMO AI can detect the small difference and has the ability to cross steps of 10cm.

Road unevenness detection

Road unevenness detection: We have been developing a wheelchair robot and a driver assist technology system for the elderly. The conventional wheelchair is difficult to get over small difference (about 30 [mm]) in level, and it sometimes makes a falling accident. To prevent the accident, the capability of detecting the small difference is required.

The system uses the Kinect V2 sensor as shown in Fig.8, and the second system uses the 3D Rolling Lidar developed by demura.net as shown in Fig.9. As far as our knowledge, it is the most advanced technology to detect small difference in level for a wheelchair size robot. Developing the small difference level detector for a commercial wheelchair and a mobility scooter for elderly, and applying the technology to a human support robot is our near future work.

Specific person detection in the outdoor environment

Detection of a specific person is not easy especially in the outdoor environment. We have developing the detection system that is the cascade deep neural network of Yolo and AlexNet. Firstly, Yolo is one of the state arts of deep neural network, and it can segment regions and classify objects. However, the classification ability is not so good. Thus, we propose a cascade of Yolo and other CNNs such as AlexNet, Goog-LeNet. Fig. 8 shows the results of Yolo and AlexNet. The system is very robust in the outdoor environment.

The results of road unevenness detection of 30 [mm] difference in level

Detection of a specific person in the outdoor environment

Conclusions and future work

This paper has described the main features of the KIT Happy Robot team that is designed with the goal of taking care of children, elderly, and for persons feel lonely living alone. The design concepts are kawaii, simplicity, safety, and usability. Thus, the robot is suitable for not only research, but also for education.

Our research goal is ``Making the World Happy by Making a Kawaii Robot.'' We are using the Robocup@Home challenge as a basis for the robot and working toward completing more important tasks for those persons in real-life situations. In the near future, we are planning to test the picture book reading application using Happy Mini in a real kindergarten.

Robot Description

Hardware

Base: Kobuki base (diferential drive), 0.7m/s max speed.

Torso: The commercial electric extendable cane, the KOZUECHAN (ITK Co. Ltd).

Head: Lovely face and some sensors is mounted on the head.

Computer: Thinkpad T450. CPU: Core i7 (Intel), GPU: 940m (NVidia)

2D LIDAR: UTM-30LX (Hokuyo Automatic)

RGB-D sensor: RealSense (Intel)

Omni camera: PIXPRO 4KVR360 (Kodak)

Gun microphone: CS-3e (Sanken)

Audio interface: MobilePre (M-Audio)

Microphone array: TAMAGO-03

(SYSTEM IN FRONTIER)

Ultrasonic Distance Sensor: PING (Parallax Inc.)

Robot 1

Robot dimensions: height: 9.3(min)-1.16m (max), width: 0.35m depth 0.38m

Robot weight: 9.45kg.

Arm: Mounted on torso. 4-DOFs, Maximum load: 0.5kg.

Robot 2

Robot dimensions: height: 0.96(min)-1.24m (max), width: 0.35m depth 0.37m

Robot weight: 8.15kg.

Arm: Mounted on torso. 5-DOFs, Maximum load: 0.78kg.

Software

OS: Ubuntu 14.04.5

Platform: ROS Indigo

Navigation: ROS Navigation stack

Localization and mapping: AMCL and Gmapping

Object recognition: YOLO v2 and CNN(LeNet)

Gender classification: Convolution DNN (LeNet)

Face recognition: HaarCascade and DNN

Speech recognition: Kaldi ASR

Sound source localization: HARK

Speech generation: MaryTTS

Arm control: ROS MoveIt!.

References

Kaldi Official Website. http://kaldi-asr.org/, (accessed March 12, 2017).
HARK Official Website. http://www.hark.jp/, (accessed March 12, 2017).
Redmon, J., Divvala, S. Girshick, R. and Farhadi, A.: You only look once: Unified, realtime object detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
Point Cloud Library Website. http://pointclouds.org/, (accessed December 4, 2017).
Demura, K., Komoriya Y.: A Navigation Method using the Mutual Feedback of Waypoints and Self-positions, Advanced Robotics, Vol.26, Issue14, pp.1677-1691, 2012.
Thrun, S., Fox, D. and Burgard, W.: Probabilistic Robotics, The MIT Press, 2002.
Fox, D., Burgard, W and Thrun, S.: The dynamic window approach to collision avoidance, Robotics & Automation Magazine, IEEE, Vol.4, pp.23-33, 1997.