Happy Mini 2017 Team Description Paper

Kosei Demura, Kensei Demura, Kazuki Nagashima, Koyo Enomoto, Takeaki Yamakawa, Ryo Iwasaki, Syuhei Mashimo

Kanazawa Institute of Technology, 7-1 Ohgigaoka Nonoichi 921-8501, Japan; D.K.T. Limited Liability Company, 1-14-19 Kanazawa 921-8034, Japan

http://demura.net/happymini2017 · http://happpymini.co

Abstract This paper describes our research interests and technical information of the Happy Mini team, former demura.net, for the RoboCup @Home league of RoboCup 2017. The Happy Mini team from the Kanazawa Institute of Technology and D.K.T. Limited Liability Company has developed an autonomous domestic service robot named "Happy Mini" for little child and elderly. This paper describes the basic architecture of the robot as well as present various algorithms and research contribution. Finally, this paper concludes and outlines our future research works.

1 Introduction

The happy mini team, our former team name was demura.net, has been participating in the @Home league of the RoboCup Japan Open since 2012, and participated in the RoboCup 2015 and 2016 world competition. The team got the 9th and 8th places in the competition, respectively.

The vision of the happy mini team is "Making the World Happy by Making a Kawaii Robot." "Kawaii" means cute, lovely, or charming. We think that the kawaii robot can solve problems of an aging society in Japan, robots in the @Home league should be more kawaii. Happy Mini as shown in Fig.4 is designed in the image of a little child with a lively yellow color. Happy Mini is considered to be the first kawaii designed robot in the RoboCup@Home. The software system is based on ROS, and the state of arts frameworks such as Caffe, Darknet, Digits, Hark, Kaldi and so on.

The rest of the paper is as follows. Section 2 describes the architecture of the robots, Section 3 presents the research contribution of our team. Finally, this paper concludes by exploring applications and denoting future work.

2 Architecture

2.1 Design Concept

Kawaii is the main design concept of Happy Mini that is used for childcare and conversation partner for elderly, and also for persons feel lonely living alone. Thus the exterior design is crucial, and a lovely and friendly exterior design is considered to be suitable for those persons. Happy Mini is the 3rd place in sympathy and 5th place in design rating in the @Home league in RoboCup2016[1]. Simplicity is also the concept, it reduces size, costs, troubles, and also danger. To realize the simplicity, the commercial Kobuki base is adopted as the platform. It is easily programmed and operated by ROS. Furthermore, safety and usability are also concept. Safety is the most important aspect for service robot. The robot is lightweight less than 10[kg], minimal power, and the joint parts are covered to prevent fingers cannot be inserted. It is also easily folded for transportation, and it can be brought as a carry-on baggage on an airplane.

2.2 Hardware

The platform of Happy Mini is the Kobuki base that is a low-cost mobile research base designed for education and research. The torso is the extensible, and the commercial electric extendable cane, the KODUECHAN (ITK Co. Ltd), is used. It is only 0.38 [kg], rated for up to 100 [kg], extends and retracts to 250 [mm]. A 4 (shoulder 1, elbow 1, wrist 2) DOFs arm with a gripper and was fully developed by our laboratory. The weight of the arm and hand as shown in Fig.4 and Fig.5 is less than 1000 [g]. The servo motors are not so powerful (4.7[Nm]), 2 motors are used for the shoulder. The arm can lift up an object up to 550 [g]. The hand is also redesigned to have a capability of grasping an object, from a thin pen to a box up to 150 [mm] on the ground.

Happy Mini has various sensors, such as the RGB-D sensor (RealSense, Intel), the LIDAR (UTM-30LX, Hokuyo), and the 360 degree camera (SP360 4K, Kodak). We developed a ROS driver for the camera[2].

2.3 Software

Perception The software system is based on ROS and Caffe. Several algorithms are used for an object and human face recognition. HaarCascade classifier[3], Flat Clustering filtering, and Deep Neural Networks (DNNs) are used for face recognition. Convolution DNNs (LeNet) and Nesterov's accelerated gradient[4] are used for gender classification. We trained the network using 700 samples for each category, and accuracy of classification is about 75 %.

The real-time object detection deep neural newtork, YOLO (You only look once)[5] is used for human detection as shown in Fig.1. The algorithm is very fast compared to other DNNs and robust.

Speech Recognition and Sound Localization We have developed a speech recognition and a sound source localization system. The speech recognition system uses the Kaldi[6] gstream server. It is a real-time full-duplex speech recognition server, and uses a DNN-based model for English trained on the TEDLIUM speech corpus. The sound source localization system is implemented using the robot auditory library HARK[7]. The HARK easily be programmed by GUI as shown in Fig.2. The system takes multi-channel speech waveform data from the 8ch microphone array, calculates FFT and estimates the sound source direction by the MUSIC method.

3 Research Contribution

3.1 Picture book reading

In the globalized modern society, the importance of acquiring English from early childhood is recognized. It is considered important to read a picture book in English. However, few Japanese speak English fluently. Thus, we have been developing a picture book reading application for children. We think that is one of the killer applications for a domestic service robot.

Fig.3 shows our demonstration of picture book reading in RoboCup2016. The outline of the algorithm is as follows,

STEP 1 : Input an image from a camera

STEP 2 : Book region extraction from the image

STEP 3 : Binarize the region using the adaptive Gaussian binarization

STEP 4 : Text line segmentation

STEP 5 : Image to text reasoning using the Bidirectional LSTM neural network.

STEP 6 : Spelling correction

STEP 7 : Calculate likelihood based on character information of sentences and excluding non sentences

STEP 8 : Output audio using a text-to-speech engine, and go to STEP 1.

3.2 Road unevenness detection and robust navigation

The first author's laboratory has been developing autonomous wheelchairs. The conventional wheelchair is difficult to get over small difference (about 30 [mm]) in level, and it sometimes makes falling accidents. To prevent the accidents, the capability of detecting the road unevenness is required.

The author has been developing the system whose ability is detecting 30 [mm] difference in the range of 0.8 [m] from the wheelchair using the Kienct V2 sensor. Down-sampling, removing outlier, smoothing, detection of a plane, and removal of the plane are used for processing the point cloud data from the sensor. The Kinect V2 can be used in an outdoor environment. This method is tested in the Tsukuba Challenge 2015[8], and its effectiveness has been proved in the real world.

The team developed a robust navigation algorithm that has a capability of solving the significant problem of the waypoint-based navigation that is the positions of the waypoints can be located in unreachable areas due to errors in self-localization and the map[9].

4 Conclusion

This paper has described the main features of Happy Mini that is designed with the goal of taking care of children, elderly, and for persons feel lonely living alone. The design concepts are kawaii, simplicity, safety, and usability. Thus, the robot is suitable for not only research, but also for education.

Our research goal is "Making the World Happy by Making a Kawaii Robot." We are using the Robocup@Home challenge as a basis for the robot and working toward completing more important tasks for those persons in real-life situations. In the near future, we are planning to test the picture book reading application using Happy Mini in a real kindergarten.

Happy Mini description

Harware

Happy Mini is designed for childcare and conversation partner for elderly. Specifications are as follows:

• Base: Kobuki base (differential drive), 0.7m/s max speed.
• Torso: The commercial electric extendable cane, the KODUECHAN (ITK Co. Ltd).
• Arm: Mounted on torso. 4 DOFs, Maximum load: 0.6kg.
• Head: Lovely face and RealSense is mounted on the head.
• Computer: Thinkpad T450. CPU: Core i7 (Intel), GPU: 940m (NVidia)
• Robot dimensions: height: 1.1(min)-1.5m (max), width: 0.32m depth 0.32m
• Robot weight: 10kg.

Also our robot has the follwoing sensors:

• 3D LIDAR: YVT-X002 (Hokuyo Automatic) – 2D LIDAR: UTM-30LX (Hokuyo Automatic)
• RGB-D sensor: RealSense (Intel)
• Omni camera: PIXPRO SP360 4K (Kodak)
• Gun microphone: CS-3e (Sanken) – Audio interface: MobilePre (M-Audio)
• Microphone array: TAMAGO-03

Software

For our robot we are using the following software:

• OS: Ubuntu 14.04.5 – Platform: ROS Indigo
• Navigation: ROS Navigation stack
• Localization and mapping: AMCL and Gmapping
• Object recognition: YOLO v2.
• Gender classification: Convoluution DNN (LeNet)
• Face recognition: HaarCascade and DNN
• Speech recognition: Kaldi ASR – Sound source localization: HARK
• Speech generation: MaryTTS
• Arm control: ROS MoveIt!.

References

1. Jahn, E.:RoboCup WM 2016, Leipzig @home liga design and sympathy rating, DOI: 10.13140/RG.2.2.17019.11042, 2016.
2. kodak360cam node, https://github.com/demulab/ kodak360cam node, (accessed March 12, 2017).
3. Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features, IEEE CVPR, 2001.
4. Nesterov, Y.: Smooth minimization of non-smooth functions, Mathematical programming, Vol.103, No.1, pp.127-152, 2005.
5. Redmon, J., Divvala, S. Girshick, R. and Farhadi, A.: You only look once: Unified, real-time object detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
6. Kaldi Official Website. http://kaldi-asr.org/, (accessed March 12, 2017).
7. HARK Official Website. http://www.hark.jp/, (accessed March 12, 2017).
8. Yuta. S, Mizukawa, M., Hashimoto, Tashiro, H., and Okubo, T.: Tsukuba Challenge 2009 – towards robots working in the real world: Records in 2009", Journal of Robotics and Mechatronics, Vol.23, No.3, pp.201–206, 2011.
9. Demura, K., Komoriya Y.: A Navigation Method using the Mutual Feedback of Waypoints and Self-positions, Advanced Robotics, vol.26, pp.1677-1691, 2012.