Gentlebots Team Description Paper

Francisco Martín, Francisco J. Rodríguez, Vicente Matellán

Rey Juan Carlos University; Luxembourg University; León University

http://gentlebots.robotica.gsyc.es/ · http://gsyc.urjc.es/~fmartin/gentlebots/

Abstract This report aims to show the capabilities of the Spanish team named Gentlebots gathered to participate in the Social Standard Platform League of the RoboCup@Home competition. Our main line of research is focused on the classic problems of mobile robotics related to navigation, auto-localization and mapping. Our goal is to get a social robot that can safely move through an environment populated with humans and interact with them in long-term operations. To this end, our latest research is a long-term mapping and navigation system that allows fast and easy deployment in domestic environments from an architectural map made up only of walls and doors. We are involved in projects that aim to help dependent people in their homes. Competitions such as the ERL (European Robotics League), which we already participate as organizers, and RoboCup@home offer us an excellent testbend for our research.

1 Introduction

Our team is composed by two universities with a great tradition in participating in robotics competitions. The Group of Robotics of the Rey Juan Carlos University participated in several RoboCup editions: 5 RoboCup (2 in 4-legged, 3 SPL), 5 German Open (4 4-legged and 1 SPL), 1 Dutch Open (4-legged), 1 Rome Cup (SPL) and 1 Latin American Open (4-legged). Between 2005 and 2013 our efforts were focused in Robot Soccer. Since 2012 we started to pay attention to @Home competitions thought the Watermelon Project. This project was initiated at at the University León's Robotics Group in 2012. In 2013, the Group of Robotics of the Rey Juan Carlos University joined the team. Together, we participated in 4 RoCKIn (camp and competition @Home) and 2 ERL (@Home) and the RoboCup@Home 2016 at Leipzig.

The scientific production related to these participations is summarized in 5 doctoral theses, 11 articles in high impact journals, and more than 30 articles in national and international congresses. It is a great production for a team that has never exceeded 10 members at a time.

Our team is focused on software development that allows robots to exhibit intelligent behaviors. For this reason, the majority of the participations in competitions have been in leagues whose robot is standard for all the participants of the league, where the focus is in the programming of the algorithms. However, Watermelon Project began with the development of a wooden robot called MYRABot [5] (Figure 5). In the last edition of RoboCup we participated with RB-1 [2], whose manufacture was carried out by Robotnik. Since last year, with the acquisition of two Pepper robots, we want to return to the standard leagues of RoboCup, this time within RoboCup@Home.

2 Innovative technology and scientific contribution

Our main contributions during our participation in @Home competitions are summarized in the following research lines:

2.1 Navigation

Starting from the standard ROS navigation [8] package, we have modified the map and location system to have a system that, based on a map of walls and doors, adds the permanent objects that the robot perceives during its operation. This is called long-term navigation [3], and is based on the fusion of several maps that reflect the presence of obstacles over time.

Our approach builds a static map starting from the construction plans of a building. A long-term map is started from the static map, and updated when adding and removing furniture, or when doors are opened or closed. A short-term map represents dynamic obstacles such as people. This approach is appropriate for fast deployment and long-term operations in office or domestic environments, able to adapt to changes in the environment.

This method was successfully used in RoboCup 2006 at Leipzig. Using this navigation system, our robot was able to plan paths incorporating furniture, detecting people and replaning when doors are suddenly closed in front of it, as you can see in the videos provided for qualification.

We plan to use an improved version of this navigation system in Pepper. In addition, we will use 3D mapping and perception, as can be seen in Figure 2.

2.2 Behavior-based Architecture

We have reused BICA [1], the behavioral architecture developed for our participation in robot soccer. We have implemented this architecture within a ROS package, in which components are executed concurrently in a hierarchical way to generate complex behaviors. The lower level components are reactive, and the higher level components are implemented as state machines.

The lower layer corresponds to ROS, and is in charge of hardware management. NaoQi driver is a ROS interface to access to the sensors and actuators equipped in the robot Pepper. We have managed to run this software inside the robot in a native way, both in its ROS version and in its own implementation in ROS2. The intermediate layer provides the robot skills in order to carry out specific duties (perception, navigation,...). We also plan to access to NaoQi functionality using a ROS nodes as proxies of NaoQi modules.

2.3 Context awareness

Context-awareness [7] is a fundamental component in human robot interaction. Understanding the user activity context the robot improves the overall decisionmaking process. This is because, given a context, the robot can reduce the set of feasible actions. We consider that using an on-board microphone and gathering environmental sounds we can improve the context recognition.

For this purpose, we have designed, developed and tested a computational lightweight Environment Recognition Component (ERC). In this iteration our ERC is able to recognize a set of four home bells. This component provides information to a Context-Awareness Component (CAC) that implements a hierarchical Bayesian network to tag user's activities based on the American Occupational Therapy Association (AOTA) classification.

2.4 Object and people detection

We have used PCL (Point Cloud Library) to detect objects from the data captured by RGB-D cameras. We are working on a perception system based on Deep Learning using YOLO2 [13] to detect objects and people in images. This software is a C/C ++ implementation of darknet, which is a convolutional neural network. This software is fully integrated in ROS through the darknet ros package [14].

We run darknet ros using the images from the robot's camera. Its output is a set of bounding boxes on the image. The 2D image is registered with the image of the RGBD camera. After applying a distance-based algorithm, we obtain the 3D position of those pixels belonging to the detected object or persons. This information is used to update the probability maps of objects and people. These maps, one for each detected category, is an octomap [4] that represents the probability of finding an element of that category in space. Those positions where detection occurs, increase their probability, while the rest of the positions decreases their probability in time. The speed of "forgetting" depends on whether a category belongs to objects of a static nature (furniture) or dynamic (people).

These maps are coordinated with the maps used for navigation, so this probabilistic memory can be used to navigate to the position where an element is likely to be found. In Figure 4 you can see what the darknet output is, how we detect chairs in our laboratory, and how these chairs are incorporated into this probabilistic memory. Note that below the octomap is the map of the environment that the robot uses to navigate.

3 Approaches used to solve RoboCup@Home challenges

In this section we will describe how we plan to solve the challenges in the stage I of the RoboCup@Home. In other stages we will work in enhancements of the capabilities used in stage I to accomplish task in other stages. For Open Challenge we plan to test the robot receptionist as a demo for the project in which Intelligent Robotics (our spin-off) is involved. Also, Tour Guide is another project that we are working on in this spin-off.

We will use the tablet of the robot to control the operation of the robot. The robot will show an interface that will allow to select the challenge that the robot must perform. During the test, you can make an interface that allows you to control the steps of the test, if necessary, or a way to cancel and restart the challenge.

3.1 Robot Inspection

In this test, the interface on the tablet will show each of the skills that will be evaluated. A high level component in BICA that implements a FSM, with a state for each test, is enough for this challenge. In particular:

– We consider that we have resolved safe navigation, as shown in Section 2.1. Local planner of the navigation module makes imposible to collide with a person.

• Detecting custom containers will be performed using darknet, trained with captures of the object made in the Setup&Preparation stage.
• For voice generation we plan to use the standard NaoQi module for speech.
• For voice recognition we have test standard NaoQi modules and it works quite well. We are testing another approach using the Google Speech API, if connection is available.

3.2 Cocktail Party

• We consider safe navigation resolved, as stated previously.
• We plan to detect people in the arena using the approach described in section 2.4. We only consider people detected inside the area of the arena. We can do it because the reference frame of the detection maps and the navigation map are coordinated.
• We will use the NaoQi modules to retrieve the source of sounds.
• For voice recognition we use standard NaoQi modules and Google Speech API.
• In this challenge, it will be necessary to combine components that implements FSMs and PDDL planners to accomplish complex tasks.
• An alternative interface will be shown in tablet if speech is not correctly processed.

3.3 General Purpose Service Robot

• Navigation resolved, as stated previously.
• The commands will be processed using standard NaoQi modules and Google Speech API. An alternative interface will be available in the tablet. This will define a task which will be processed by the PDDL planner.

3.4 Help-me-carry

• Navigation resolved, as stated previously.
• We plan to use the Fast Training approach to select the person to follow in each step.
• Small and 3D objects fits well with our 3D navigation system. Our mapping system works really well when doors re-open or obstacles dissapear (see qualification videos of our participation in RoboCup).

3.5 Speech and person recognition

• People will be detected using the approach described in section 2.4.
• Sound source detection will pe carried out using the NaoQi modules designed to this task.
• Speech will be processed using standard NaoQi modules and Google Speech API.

4 Material resources

We have 2 Pepper robots, and we will bring 1 Kobuki robot to the RoboCup 2018 in Montreal as a support to test algorithms in parallel. We have two laboratories, one of which is certified for @Home competitions.

5 Contributions: Source Code Availability

We have different GitHub available to reach our solutions. As individuals it is possible to see our developments in:

• https://github.com/fmrico
• https://github.com/FranLera

As a team our developments are available after each competition in the next GitHub repositories:

• https://github.com/fmrico/WaterMellon
• https://github.com/Robotica-ule/MYRABot

To share information, methodologies and developments is one of our principles. Former members of our team have contributed with manuals in other fields as in PCL

– http://robotica.unileon.es/mediawiki/index.php/PhD-3D-Object-Tracking

6 Conclusions and Future Work

This TDP described the main developments of our team for the 2018 RoboCup@home competition. We have presented our trajectory in @Home competitions, work done and published, and modules we are working on. We are sure to successfully face the challenges for our league.

We are committed to open source, and all our work is available before, during and after the competition. We are also active members of @Home competitions community, organizing ERL editions and as OC of RoboCup@Home.

The skill set presented will allow us to introduce us at RoboCup@Home 2018 competition.

7 Robot's Hardware and Software Description

(No introductory text)

7.1 Hardware

The RoboCup@Home Social Standard Platform competition will be run with the standard platform Softbank Robotics Pepper.

7.2 Software

All software that will be developed will be based on ROS Kinetic as development framework, which runs natively on board the robot. Additional ROS nodes will be developed in order to access the NaoQi functionalities on speech recognition and generation, as well as others that we consider necessary.

– Vision: ork, findobject, self developments using Deep Learning (darknet ros-YOLO2 and tensorflow).

– Dialogue: Pocketsphinx, NaoQi

– BellRecognition: pyaudio, tensorflow

– Manipulation: MoveIt, NaoQi.

– Navigation: Ros stack, self developments made on mapping and localization.

– Behavior generation: BICA (self development) and ROSPlan.

– Interaction by tablet: HTML5 web pages and RobotWebTools

References

1. Agüero, C.E., Canas, J.M., Martín, F., Perdices, E.: Behavior-based iterative component architecture for soccer applications with the nao humanoid. In: 5th Workshop on Humanoids Soccer Robots. Nashville, TN, USA (2010)
2. Espert, A.: Robotnik announces the rb-1, a new ros based mobile manipulator, http://www.ros.org/news/2015/02/ robotnik-announces-the-rb-1-a-new-ros-based-mobile-manipulator.html
3. Ginés, J., Martín, F., Matellán, V., Lera, F.J., Balsa, J.: Dynamics maps for longterm autonomy. In: IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) (2017)
4. Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots (2013), http://octomap.github.com, software available at http: //octomap.github.com
5. Lera, F.J., Casado, F., Rodríguez, C., Olivera, V.M.: Building low-cost mobile manipulation platform for rockin@ home competition. In: XV Workshop of physical agents: book of proceedings, WAF 2014, June 12th and 13th, 2014 León, Spain. pp. 41–50 (2014)
6. Lera, F.J.R., Llamas, C.F., Angel Manuel Guerrero, Matellán, V.: Cyber-security of robotics and autonomous systems: Privacy and safety. Robotics - Legal, Ethical and Socioeconomic Impacts (2017)
7. Lera, F.R., Rico, F.M., Olivera, V.M.: Deep learning and bayesian networks for labelling user activity context through acoustic signals. In: IWINAC. International Work-conference on the Interplay between Natural and Artificial Computation (2017)
8. Marder-Eppstein, E., Berger, E., Foote, T., Gerkey, B., Konolige, K.: The office marathon: Robust navigation in an indoor office environment. In: International Conference on Robotics and Automation (2010)
9. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source roboft operating system. In: ICRA Workshop on Open Source Software (2009)