View Original PDF

Team eR@sers[DSPL] (Toyota HSR) 2017 Team Description Paper

H. Okada, H. Yokoyama, T. Ogata, T. Inamura, N. Iwahashi, K. Sugiura

Tamagawa University; Waseda University; National Institute of Informatics; Okayama Prefectural University; National Institute of Information and Communications Technology

https://sites.google.com/site/erasers2050/home/

Abstract Team eR@sers has taken part in RoboCup@Home since 2008. 2008 was the first year of the eR@sers. The eR@sers achieved a first place at RoboCup 2008 ,2010 and second place RoboCup 2009, 2012. and its social robot HSR obtained the @Home Innovation Award in 2016. Our team, eR@sers were adopted for RoboCup@Home Standard Platform(HSR) User Teams. We have imploved the ability of robots with various techniques, which are going to be applied to other robot systems or social IT systems. We introduce them and our latest research briefly in this description.

1 Team Summary

(No introduction text before subsections)

1.1 History

The Japanese Robot Team eR@sers(erasers) is the result of a joint effort of four Japanese research groups:

Tamagawa University:the group of the College of Enginerring at Tamagawa University of Tokyo in Japan that is involved in the world championship RoboCup competitions in Four-legged since 2005. At the RoboCup 2006 Bremen , the team, FC Twaves, got the best results(best 16) in the team that participated from Japan. The members at Tamagawa University are interested in a compliant human-machine interaction architecture that is based on the machine intention recognition of the human. This work is motivated by the desire to minimize the need for classical direct human-machine interface and communication.

Waseda University,National Institute of Informatics ,Okayama Prefectural University and National Institute of Information and Communications Technology that are involved in the research of the computational mechanism which enable robots to learn the communication by language and actions through natural interaction with human. Based on the success of the preliminary challenge of the @Home Simulation in RoboCup Japan Open 2013 Tokyo, and the demonstration in the international RoboCup 2013 Eindhoven, the proposal of a new RoboCup @Home Simulation challenge has obtained the recognition of the international RoboCup committee and held the demo challenge in the RoboCup 2015 China in July 2015.

1.2 Focus of research/research interests

We mainly focus on the adaptability to the environmental changes, and on the integration between the sensory-motor data and symbolic representation, utilizing only the neuro-dynamical model.

1.3 Re-usability of the system for other research groups

All developed functions could be packed in ROS modules.

1.4 Applicability of the robot in the real world

Almost all training data would be real data and the system is performed and evaluated in the real environment

1.5 3rd party robot's software

Ubuntu14.04 + ROS Indigo Tensorflow 1.0

2 Innovative technology and scientific contribution

(No introduction text before subsections)

2.1 Learning non-parametric policies as random variable transformations

Learning how to act under uncertainty is a central problem in the field of machine learning. One of major approaches to attack this problem is policy gradient, which explicitly represents the policy and attempts to find the optimal one. Although this makes it straightforward to generate continuous action value, the policy is confined to a particular family of distributions such as normal distributions due to the difficulty of drawing action values from arbitrary probability distributions. To get rid of this limitation, we propose a method to learn nonparametric policy without any concern about generating action values.

Our approach, shown in Fig.1, is to find a transformation from random variable nt (following a normal distribution for simplicity), whose distribution is known, to one that follows desired distribution. Consider a random variable transformation from nt to the action value at depending on the state value st, namely,

$$a_t = f(s_t, n_t; \theta), \tag{1}$$

where θ is the parameter of the function approximator. In this case, the form of the distribution of at can vary depending on st and θ. In particular, the distribution can be multimodal if the function approximator is sufficiently expressive. Once one find f such that at follows desired distribution, samples of the distribution can easily be drawn by calculating f.

Fig. 1. A schematic illustration of our approach. Action value a is calculated from the state value s and noise n. The function f of them is learned to acquire a desired distribution of a. The optimization is performed with respect not to the value of n but to the distribution of n.
Fig. 1. A schematic illustration of our approach. Action value a is calculated from the state value s and noise n. The function f of them is learned to acquire a desired distribution of a. The optimization is performed with respect not to the value of n but to the distribution of n.
Fig. 2. A numerical experiment. The input noise n was drawn from N(0, 1), which is indicated by dashed curves (the horizontal axes are shared by n and a). The histograms show the distribution of a = f(n). The solid curves show the reward function r = 0.7e *<sup>−</sup>*(a+1)2/<sup>4</sup> + 0.7e *<sup>−</sup>*(a*−*1)2/<sup>4</sup> − 1.5e *−*a <sup>2</sup>/2 .
Fig. 2. A numerical experiment. The input noise n was drawn from N(0, 1), which is indicated by dashed curves (the horizontal axes are shared by n and a). The histograms show the distribution of a = f(n). The solid curves show the reward function r = 0.7e *−*(a+1)2/4 + 0.7e *−*(a*−*1)2/4 − 1.5e *−*a 2/2 .

2.2 Language-motion translation

We will enhance our recurrent neural net model for language-motion translation. The model is based on the sequence-to-sequence model and it is trained with sentences and corresponding motions of robot (??). The model enable the robot to handle the ambiguity and synonymy in the sentences, such as logical words "not", "and", and "or".

Fig. 3. Translation from a sentence to a motion by RNN
Fig. 3. Translation from a sentence to a motion by RNN

2.3 Object handling

We will enhance our deep learning model for object recognition and motion generation. The model consists of multiple deep auto-encoders with the time sequence input of multi-modal information (4). The model directly control all joints of the robot which handles the object by using raw image and/or sound.

Fig. 4. Cloth folding task based on deep learning model
Fig. 4. Cloth folding task based on deep learning model

2.4 RoboCup@Home Simulation

Research on high level human-robot interaction systems that aims skill acquisition, concept learning, modification of dialogue strategy and so on requires large-scaled experience database based on social and embodied interaction experiments. However, if we use real robot systems, costs for development of robots and performing many experiments will be too huge. If we choose virtual robot simulator, limitation arises on embodied interaction between virtual robots and real users. We thus propose an enhanced robot simulator that enables multiuser to connect to central simulation world, and enables users to join the virtual world through immersive user interface. As an example task, we propose an application to RoboCup@Home tasks.

Another function that can be simulated is using natural language and gesture instruction to recognize which object is being referred to by the user. A user might ask the robot to move and/or manipulate an object by saying something like,"Please bring that dish to the dining table"while pointing to the dish. If the pointing and/or speech are vague, the robot should be able to ask appropriate questions to remove the uncertainty. Such dialogue management is a high-level interaction function inherent in high-level HRI.

3 Contribution for RoboCup@Home

Starting from 2006, RoboCup@Home has been the largest international annual competition for autonomous service robots as part of the RoboCup initiative. The challenge consists of a set of benchmark tests to evaluate the robots 'abilities and performances in a realistic non-standardized home environment setting.

Fig. 5. Clean-up task
Fig. 5. Clean-up task
Fig. 6. RoboCup@Home Education initiative to promote service robot development for entry level audience in RoboCup@Home
Fig. 6. RoboCup@Home Education initiative to promote service robot development for entry level audience in RoboCup@Home

4 The contents of the web site

Our relevant publications, technical reports, as well as videos and pictures ar available in :

Official website:

https://sites.google.com/site/erasers2050/home/

Photos and Videos of the robot:

https://sites.google.com/site/erasers2050/photos-movies/

5 External computing

We will use "External devices" or" Cloud services "or" Internet API 's "as listed below

Desktop Computer

  • 6th generation Intel Core i7-7700K processor 4.4 GHz
  • 32GB RAM DDR4
  • Windows 10 Home, Ubuntu14.04LTE

ROSPEEX

ROSPEEX is a cloud-based multilingual communication package for ROS (Robot Operating System). (For more information, visit http://rospeex.org/top/)

Microsoft Cognitive Services API

We use Microsoft Cognitive Services API.

(For more information, visit https://azure.microsoft.com/ja-jp/ services/cognitive-services/)

IBM Watson Developer Cloud

We use IBM Watson Developer Cloud. (For more information, visit https://www.ibm.com/jp-ja/ marketplace/cognitive-application-development)

NVIDIA GeForce NOW

(For more information, visit https://www.nvidia.com/en-us/ shield/games/#geforcenow/)

References

  1. Hiroki Yokoyama and Hiroyuki Okada: Learning non-parametric policies as random variable transformations, The 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016), 2016.
  2. Tatsuro Yamada, Shingo Murata, Hiroaki Arie, and Tetsuya Ogata: Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction, Frontiers in Neurorobotics, 2016.
  3. Tatsuro Yamada, Shingo Murata, Hiroaki Arie, and Tetsuya Ogata, " Logically Complex Symbol Grounding for Interactive Robots by Seq2seq Learning with an LSTM-RNN, " In the Thirtieth Annual Conference on Neural Information Processing Systems (NIPS2016), Acceptance Rate for demonstration: (36.4
  4. Kuniaki Noda, Hiroaki Arie, Yuki Suga, and Tetsuya Ogata: Multimodal Integration Learning of Robot Behavior using Deep Neural Networks, Robotics and Autonomous Systems, Vol. 62, No. 6, pp. 721-736, 2014.
  5. Pin-Chu Yang, Kazuma Sasaki, Kanata Suzuki, Kei Kase, Shigeki Sugano, and Tetsuya Ogata: Repeatable Folding Task by Humanoid Robot Worker using Deep Learning, IEEE Robotics and Automation Letters (RA-L), accepted, 2017.
  6. Tetsunari Inamura, Satoshi Unenaka, Satoshi Shibuya, Yukari Ohki, Yutaka Oouchida, Shin-ichi Izumi, Development of VR platform for cloud-based neurorehabilitation and its application to research on sense of agency and ownership, Advanced Robotics, Vol. 31, Iss. 1-2, pp. 97-106, 2017.
  7. Tetsunari Inamura, A Cloud Based VR Platform for Sharing Embodied Experience in HRI, The 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016), TuK1, 2016.