SPQReL 2018 Team Description Paper

M.T. Lázaro, L. Iocchi, D. Nardi, J. P. Fentanes, M. Hanheide

Dept. of Computer, Control and Management Engineering, Sapienza University of Rome, Via Ariosto, 25, 00185 Roma, Italy; School of Computer Science, University of Lincoln, Brayford Pool, Lincoln, LN6 7TS, England

https://sites.google.com/dis.uniroma1.it/spqrel · https://coaches.greyc.fr/ · http://strands.acin.tuwien.ac.at/ · https://www.ald.softbankrobotics.com/en/cool-robots/pepper · https://github.com/LCAS/spqrel_navigation · https://github.com/strands-project/strands_navigation · https://github.com/mtlazaro/cg_mrslam · https://github.com/strands-project/strands_perception_people · https://github.com/LCAS/spqrel_tools/tree/master/slu4p · https://github.com/iocchi/PetriNetPlans · https://cloud.google.com/speech · https://azure.microsoft.com/en-us/services/cognitive-services/face

Abstract SPQReL is a joint team of Sapienza University of Rome, Italy, and the University of Lincoln, UK, formed by two research groups that have developed several solutions for the deployment of social robots in public environments. The main research objectives are focussed on the integration between Artificial Intelligence and Human-Robot Interaction (HRI) and, in particular, on robot planning and learning, long-term autonomy, and HRI with untrained users.

Our main motivations to participate in the RoboCup@Home Social Standard Platform League are: 1) the development of a joint effort to integrate and consolidate the solutions developed by the two groups (and in complementary European projects), 2) the adaptation of these solutions to the RoboCup@Home tasks, and the dissemination and release of these solutions to the RoboCup@Home community.

Introduction

SPQReL (pronounced as "Sparkle") is a joint team formed last year by two research groups: 1) Cognitive Cooperating Robots (RoCoCo) Lab at Sapienza University of Rome, Italy and 2) Lincoln Centre for Autonomous Systems (L-CAS) at University of Lincoln, UK.

The main research objective of the joint team is to foster further collaboration for developing effective solutions for social and service robots in public spaces. The two research groups gained a lot of experience in this field with the participation to several projects related to this topic. In particular, the recent projects COACHES and STRANDS have developed complementary components for social robots in public environments. The main goals of the joint team are: i) integrate the individual outcomes of the involved projects accomplished by the two research groups into a more functional and robust social robot, ii) adopt these solutions for the RoboCup@Home environment (i.e., to solve RoboCup@Home tasks), iii) release and disseminate outcomes stemming from this initiative and the projects involved readily to the RoboCup@Home community.

Our joint team participated in RoboCup@Home SSPL 2017 in Nagoya, gaining the 3rd place, demonstrating the value and success of our collaboration.

Scientific contributions and results

In this section, we discuss the main recent scientific contributions achieved by the two research groups and the research topics relevant to the RoboCup@Home competitions in general and we outline concrete scientific results obtained by our team during the last year in the context of RoboCup@Home SSPL as a result of our collaboration.

General objectives

Planning and plan execution. The deployment of robots in populated environments interacting with non-expert users requires facing many sources of uncertainty during task execution such as incomplete information about the environment or unpredictable behaviours coming from humans. Planning and plan execution under such uncertainties is also an important problem to be addressed within the RoboCup@Home competition and in this context, both Sapienza and Lincoln partners have recent research results.

In [1], we propose a method using conditional planning for generating and executing short-term interactions, in [2], we propose an extension to the Hierarchical Agent-based Task Planner (HATP) for the automatic generation of conditional plans that enables humans and robots to negotiate some aspects of the collaboration online during the execution of the plan and, in [3], task failure is handled intelligently by combining different types of robot's knowledge to solve the problem of task planning and execution under uncertainty and in open worlds, explanation of task failure and verification of those explanations.

Human-Robot Interaction. Social robots deployed in large public spaces have to carry out short-term interactions with many unknown people. In order to provide a better user experience, personalized multi-modal interactions have shown to be more effective. In this context, [4] presented a module for HRI based on explicit representation of social norms that provides a high degree of variability in the personalization of the interactions, maintaining easy extendibility and scalability.

Generating appropriate robot behaviours during the interaction it is also a key factor to achieve successful interactions. In [5], the problem of maintaining Human-Robot Spatial Interactions (HRSI) is studied from the point of view of the Proxemics, where distances between the agents are included into a probabilistic model based on a Qualitative Trajectory Calculus.

Finally, analysing the performance of an HRI system in order to improve the interactions requires a systematic approach. In [6], a method is proposed to jointly analyse system level and interaction level in an integrated manner. The approach allows to trace back patterns that deviate from prototypical interaction sequences to the distinct system components of the robot.

Long-term autonomy. One of the main goals of the RoboCup@Home is to develop a system able to robustly navigate in dynamic environments subject to changes and unpredictable situations. In this context, [7] presented a localization and mapping system based on a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. The proposed representation improves localisation accuracy and the efficiency of path planning.

In [8], we present an approach for topological navigation of service robots in dynamic indoor environments this approach uses a topological representation of the environment that simplifies definition of navigation actions, and is augmented with a spatio-temporal model that specifically represent changes that stem from events in the environment, which impact on the success probability of planned actions which allows the robot to predict action outcomes and to devise better navigation plans.

In [9], we have also shown how better HRI can be facilitated by exploiting long-term spatio-temporal experience, similar to the approached above, but directly linking long-term autonomy with setting goals for a mobile robot.

In populated environments, the ability to be able to predict the directions people are heading is useful for robots to plan suitable path. The machine learning method in [10] allows to learn a model for such predictions from long-term experience.

Scientific Results

The scientific achievements presented in the previous section have been successfully applied on the robotic platforms available for both research groups in the framework of the COACHES and STRANDS projects (see some examples on Fig. 1). Furthermore, concrete scientific results have been obtained in the context of RoboCup@Home, which are highlighted below:

Localization and navigation. A navigation system suitable for robots with low computational resources, including the Pepper robot, has been proposed. The system includes a localization system, path planning and obstacle avoidance and topological navigation. The software has been released as open source and the result of this collaboration has been published on the ROBOT2017 conference [11].

Planning and execution. The Petri Net Plans (PNP) execution framework has been extended to the NAOqi framework, to be used with the Pepper robot. Furthermore, a Blocky-based online plan design tool has been developed, which allows for easy and interactive PNP creation and debugging. We consider this tool has a potential use for educational robotics purposes.

Speech understanding. A hybrid ASR system that combines the output of the Google Speech API and Nuance, the native ASR included in the Pepper robot. The system provides a re-ranking feature that allows to prioritize transcriptions based on domain specific dictionaries. The LU4R software is used for interpretation of natural language commands and a dialogue manager based on AIML controls the dialogue flow.

System Architecture and Implementation

All the software developed by our team for the RoboCup@Home SSPL is based on NAOqi, the Pepper robot's native Operating System, and released as opensource.

Our goal is to have our system running on-board the Pepper robot without depending on external computing resources. At this moment, we can say our system is fully independent from external resources on the navigation stack and task planning generation and execution components. Even so, some of our system components are implemented using a hybrid approach that makes use of external cloud services when available, without fully depending on them. This is the case of the ASR system, that makes use of the Google Speech API if the robot has access to the cloud service, otherwise the result provided by the Nuance ASR is used.

Different processes communicate using the ALMemory (shared memory) of the NAOqi OS. We use the ALMemory system of declaration and subscription to events in a similar fashion than ROS topics.

More details about the software components that we are aiming to use during the RoboCup@Home competition are presented in the Robot's Software Description at the end of this document.

Organization of the Team

The participants of the joint team have already agreed on the management structure and sharing of technological and financial responsibility. The general team leader who will be the contact person with RoboCup@Home organization is Dr. María Teresa Lázaro. She received her PhD on January 2015 from Universidad of Zaragoza, Spain. Her thesis, supervised by Prof. José Angel Castellanos, was entitled Map building, localization and exploration for multi-robot systems. During her PhD, she was visiting student at LabRoCoCo for 8 months, under the supervision of Prof. Giorgio Grisetti. She fully joined LabRoCoCo on June 2015. Since then, she has been the technical manager of the LabRoCoCo unit in the COACHES project and the main developer and maintainer of the DIAGO robot. The development at Lincoln will be led by Dr. Jaime Pulido Fentanes, who will coordinate directly with María Teresa Lázaro to jointly decide on technical developments.

The scientific coordination is overseen by the executive board comprising Prof. Luca Iocchi, Prof. Giorgio Grisetti and Prof. Daniele Nardi from Sapienza, and Dr. Marc Hanheide and Dr. Heriberto Cuayahuitl from Lincoln. The executive board manages finances and liaison with sponsors, etc.

Each research group is formed by team members (mostly PhD students and students of the Master in Artificial Intelligence and Robotics at Sapienza University of Rome and PhD and MSc students at L-CAS) who will be coordinated by the two team leaders. Currently, the following team members are taking an active role in the preparation for the Robocup competition:

• Sapienza University of Rome. Andrea Vanzo (speech understanding).
• University of Lincoln. Sergi Molina (robot vision people perception), Roberto Pinillos (robot vision - object recognition).

The agreed and established management approach comprises: focus on scientific research (publications and releasing working software trumps winning the competition), bottom-up approach (develop robustly working, integrated components), benchmarking and performance measures (quantitative evaluation of progress and iterative development), internal collaboration and competition (assigning the same module to different sub-teams in the team and compare the performance), good software engineering practices (inherited from our projects).

Conclusions and future work

The SPQReL RoboCup@Home team is founded on the solid basis of two research groups that have developed effective solutions for social robots in public environments. We believe that this joint effort will further improve the quality of the scientific approaches and the robustness and effectiveness of the developed software. The challenging scenario of RoboCup@Home competitions will allow a proper evaluation and benchmarking of the developed approaches and the results will be beneficial for all the RoboCup@Home community.

Robot's Hardware and Software Description

The RoboCup@Home Social Standard Platform competition will be run with the standard platform Softbank Robotics Pepper. A list of software components that we plan to use in the team is shortly summarized here and described in more details in the website (which includes also link to publicly available releases).

Navigation, localization and mapping

• spqrel_navigation, a NAOqi-based navigation system.
• strands_navigation, a ROS-based navigation system.
• cg_mrslam, a ROS-based single and multi robot 2D SLAM system.

People detection and tracking

• strands_perception_people, ROS-based component developed in the STRANDS project and that we aim to port and integrate for the Pepper robot.

Speech understanding

• SLU4P: A Spoken Language Understanding for Pepper.

Plan execution and monitoring

• Petri Net Plans: NAOqi library.

Equipment to be used during the competition

Additionaly we intend to use the following external resources:

• 1 or 2 laptops for external computation
• Cloud services:
  • Google Cloud Speech API
  • Microsoft Face API

References

1. V. Sanelli, M. Cashmore, D. Magazzeni, and L. Iocchi. Short-term human robot interaction through conditional planning and execution. In Int. Conf. on Automated Planning and Scheduling (ICAPS), 2017.
2. E. Sebastiani, R. Lallement, R. Alami, and L. Iocchi. Dealing with on-line humanrobot negotiations in hierarchical agent-based task planner. In Int. Conf. on Automated Planning and Scheduling (ICAPS), 2017.
3. M. Hanheide et al. Robot task planning and explanation in open and uncertain worlds. Artificial Intelligence, 2015.
4. L. Iocchi, M. T. Lázaro, L. Jeanpierre, and A.-I. Mouaddib. Personalized shortterm multi-modal interaction for social robots assisting users in shopping malls. In Int. Conf. on Social Robotics, Paris, France, Oct 26-30 2015.
5. C. Dondrup, N. Bellotto, M. Hanheide, K. Eder, and U. Leonards. A computational model of human-robot spatial interactions based on a qualitative trajectory calculus. Robotics, 4(1):63, 2015.
6. Manja Lohse, Marc Hanheide, Karola Pitsch, Katharina J Rohlfing, and Gerhard Sagerer. Improving HRI design by applying Systemic Interaction Analysis (SInA). Interaction Studies, 10(3):298–323, dec 2009.
7. J. Pulido Fentanes, T. Krajnik, M. Hanheide, and T. Duckett. Persistent localization and life-long mapping in changing environments using the frequency map enhancement. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Daejeon, Korea, Oct. 9 - 14 2016.
8. J. Pulido Fentanes, B. Lacerda, T. Krajník, N. Hawes, and M. Hanheide. Now or later? predicting and maximising success of navigation actions from long-term experience. In Int. Conf. on Robotics and Automation (ICRA), 2015.
9. Marc Hanheide, Denise Hebesberger, and Tomas Krajnik. The When, Where, and How: An Adaptive Robotic Info-Terminal for Care Home Residents – A long-term Study. In Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), Vienna, 2017. ACM/IEEE.
10. Li Sun, Zhi Yan, Sergi Molina Mellado, Marc Hanheide, and Tom Duckett. 3dof pedestrian trajectory prediction learned from long-term autonomous mobile robot deployment data. arXiv preprint arXiv:1710.00126, 2017.
11. M. T. Lázaro, G. Grisetti, L. Iocchi, J. P. Fentanes, and M. Hanheide. A lightweight navigation system for mobile robots. In Third Iberian Robotics Conference (ROBOT), 2017.