View Original PDF

SPQReL 2017 Team Description Paper

M.T. L´azaro, L. Iocchi, D. Nardi, M. Hanheide, J. P. Fentanes

Dept. of Computer, Control and Management Engineering Sapienza University of Rome Via Ariosto, 25, 00185 Roma, Italy; School of Computer Science, University of Lincoln Brayford Pool, Lincoln, LN6 7TS, England

http://tinyurl.com/spqrel · https://coaches.greyc.fr/ · http://strands.acin.tuwien.ac.at/ · http://wiki.ros.org/pepper · https://www.ald.softbankrobotics.com/en/cool-robots/pepper

Abstract SPQReL is a joint team of Sapienza University of Rome, Italy, and the University of Lincoln, UK, formed by two research groups that have developed several solutions for the deployment of social robots in public environments. The main research objectives are focussed on the integration between Artificial Intelligence and Human-Robot Interaction (HRI) and, in particular, on robot planning and learning, long-term autonomy, and HRI with untrained users.

Our main motivations to participate in the RoboCup@Home Social Standard Platform League are: 1) the development of a joint effort to integrate and consolidate the solutions developed by the two groups (and in complementary European projects), 2) the adaptation of these solutions to the RoboCup@Home tasks, and the dissemination and release of these solutions to the RoboCup@Home community.

Introduction

SPQReL (pronounced as "Sparkle") is a joint team formed by two research groups: 1) Cognitive Cooperating Robots (RoCoCo) Lab at Sapienza University of Rome, Italy and 2) Lincoln Centre for Autonomous Systems (L-CAS) at University of Lincoln, UK.

The main research objective of the joint team is to foster further collaboration for developing effective solutions for social and service robots in public spaces. The two research groups gained a lot of experience in this field with the participation to several projects related to this topic. In particular, the recent projects COACHES and STRANDS have developed complementary components for social robots in public environments. The main goals of the joint team are: i) integrate the individual outcomes of the involved projects accomplished by the two research groups into a more functional and robust social robot, ii) adopt these solutions for the RoboCup@Home environment (i.e., to solve RoboCup@Home tasks), iii) release and disseminate outcomes stemming from this initiative and the projects involved readily to the RoboCup@Home community.

Scientific contribution

In this section, we discuss the main scientific contributions achieved by the two research groups and the research topics relevant to the RoboCup@Home competitions and outline how they will contribute and be further developed in this context. As will be shown, the teams contribute complementary achievements so that their integration will allow for increased robustness and effectiveness of the proposed solutions. We thus believe that our RoboCup@Home project will be the basis to achieve additional novel results on the topics mentioned below.

Planning and plan execution

The deployment of robots in populated environments interacting with non-expert users requires facing many sources of uncertainty during task execution such as incomplete information about the environment or unpredictable behaviours coming from humans. Planning and plan execution under such uncertainties is also an important problem to be addressed within the RoboCup@Home competition and in this context, both Sapienza and Lincoln partners have recent research results. In [1], a practical framework based on a decision-theoretic formalism for generation and execution of robust plans for service robots is presented and, in [2], task failure is handled intelligently by combining different types of robot's knowledge to solve the problem of task planning and execution under uncertainty and in open worlds, explanation of task failure and verification of those explanations.

Furthermore, robots interacting with humans are expected to demonstrate socially acceptable behaviours and to follow social norms. However, most of the recent works in this field do not address the problem of explicit representation of the social norms and their integration in the reasoning and the execution components of a cognitive robot. In [3], we present a framework for planning and execution of social plans, in which social norms are described in a domain and language independent form.

Human-Robot Interaction

Social robots deployed in large public spaces have to carry out short-term interactions with many unknown people. In order to provide a better user experience, personalized multi-modal interactions have shown to be more effective. In this context, [4] presented a module for HRI based on explicit representation of social norms that provides a high degree of variability in the personalization of the interactions, maintaining easy extendibility and scalability. However, the design of such multi-modal interactions can be a complex and time-consuming process. [5] proposes a formalism for the description of multi-modal interactions and the methods for their automatic generation and execution. The approach is based on the use of interaction templates to facilitate the design and management of the multi-modal behaviour.

Generating appropriate robot behaviours during the interaction it is also a key factor to achieve successful interactions. In [6], the problem of maintaining Human-Robot Spatial Interactions (HRSI) is studied from the point of view of the Proxemics, where distances between the agents are included into a probabilistic model based on a Qualitative Trajectory Calculus.

Finally, analysing the performance of an HRI system in order to improve the interactions requires a systematic approach. In [7], a method is proposed to jointly analyse system level and interaction level in an integrated manner. The approach allows to trace back patterns that deviate from prototypical interaction sequences to the distinct system components of the robot.

Long-term autonomy

One of the main goals of the RoboCup@Home is to develop a system able to robustly navigate in dynamic environments subject to changes and unpredictable situations. In this context, [8] presented a localization and mapping system based on a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. The proposed representation improves localisation accuracy and the efficiency of path planning. In [9], we present an approach for topological navigation of service robots in dynamic indoor environments this approach uses a topological representation of the environment that simplifies definition of navigation actions, and is augmented with a spatio-temporal model that specifically represent changes that stem from events in the environment, which impact on the success probability of planned actions which allows the robot to predict action outcomes and to devise better navigation plans.

In [10], we have also shown how better HRI can be facilitated by exploiting long-term spatio-temporal experience, similar to the approached above, but directly linking long-term autonomy with setting goals for a mobile robot.

Scientific Results

The scientific achievements presented in the previous section have been successfully applied on the robotic platforms available for both research groups in the framework of the COACHES and STRANDS projects. During the different experiments carried out in these projects, robots have been deployed in several public and controlled environments, such as malls, exhibition events, working places, museums, hospitals, or elder care sites, where the robots navigated autonomously and interacted with different non-expert users. These scenarios (see some examples on Fig. 1) are similar to those considered for the RoboCup@Home competition and we expect to adapt and improve our current working techniques to its context.

Fig. 1. Top row: COACHES and DIAGO robots. Bottom row: STRANDS robots.
Fig. 1. Top row: COACHES and DIAGO robots. Bottom row: STRANDS robots.

System Architecture and Implementation

All the software developed by the two research groups is based on ROS and released as open-source. Some integration work has already been realized: 1) the PNP library developed by LabRoCoCo has been integrated in the STRANDS PROJECT, 2) the integration of the people tracking system developed by L-CAS lab is in progress, where laser-based people tracking is already functioning which will be robustified using vision-based people detection.

The Pepper robot ordered by our team has not been delivered yet, thus, we have been unable to test our software on the actual robot used in the competition. However, as the Pepper robot supports ROS, we believe that the porting of our software on Pepper will require a reasonable effort. Moreover, the developed software has been implemented and tested on robots functionally equivalent to Pepper (See Fig. 1). These robots have a similar sensor and devices configuration set-up to Pepper, providing common capabilities. Among others, we highlight the use of laser sensors for navigation, localization and SLAM, a top RGB-D camera for people perception and a tablet and a microphone for multi-modal humanrobot interaction. For this reasons, we believe we have all the expertise needed to adapt our software to the new platform when it will be available.

The results of this effort will be anyway released to the community even before the RoboCup@Home 2017 competition and we will grant permission to other teams to use the software ported from our projects to Pepper even for RoboCup@Home 2017. More details about the software components that we are aiming at integrating are presented in the Robot's Software Description at the end of this document.

Organization of the Team

The participants of the joint team have already agreed on the management structure and sharing of technological and financial responsibility. The general team leader who will be the contact person with RoboCup@Home organization is Dr. María Teresa Lázaro. She received her PhD on January 2015 from Universidad of Zaragoza, Spain. Her thesis, supervised by Prof. José Angel Castellanos, was entitled Map building, localization and exploration for multi-robot systems. During her PhD, she was visiting student at LabRoCoCo for 8 months, under the supervision of Prof. Giorgio Grisetti. She fully joined LabRoCoCo on June 2015. Since then, she has been the technical manager of the LabRoCoCo unit in the COACHES project and the main developer and maintainer of the DIAGO robot. The development at Lincoln will be led by Dr. Jaime Pulido Fentanes, who will coordinate directly with María Teresa Lázaro to jointly decide on technical developments.

The scientific coordination is overseen by the executive board comprising Prof. Luca Iocchi and Prof. Daniele Nardi from Sapienza, and Dr. Marc Hanheide and Dr. Heriberto Cuayahuitl from Lincoln. The executive board manages finances and liaison with sponsors, etc.

Each research group is formed by team members (mostly PhD students and students of the Master in Artificial Intelligence and Robotics at Sapienza University of Rome and PhD and MSc students at L-CAS) who will be coordinated by the two team leaders. Currently, the following team members are taking an active role in the preparation for the Robocup competition:

  • Sapienza University of Rome. Dr. Roberto Capobianco (semantic mapping), Federico Nardi (2D navigation and planning), Ali Youssef (people detection and tracking), Lun Wang (speech understanding).
  • University of Lincoln. Sergi Molina (simulation), Roberto Herrero (task design), Petra Bosilj (object vision), Jaycee Lock, (person vision).

The team will substantially benefit from the experiences in participating to RoboCup competitions of the group at Sapienza since 1998, including the participation at RoboCup@Home 2006 as members of the RoboCare team, and the experience of Dr. Marc Hanheide as a team leader in RoboCup@Home 2009 (Team ToBI). The agreed and established management approach comprises: focus on scientific research (publications and releasing working software trumps winning the competition), bottom-up approach (develop robustly working, integrated components), benchmarking and performance measures (quantitative evaluation of progress and iterative development), internal collaboration and competition (assigning the same module to different sub-teams in the team and compare the performance), good software engineering practices (inherited from our projects).

Conclusions and future work

The SPQReL RoboCup@Home team is founded on the solid basis of two research groups that have developed effective solutions for social robots in public environments. We believe that this joint effort will further improve the quality of the scientific approaches and the robustness and effectiveness of the developed software. The challenging scenario of RoboCup@Home competitions will allow a proper evaluation and benchmarking of the developed approaches and the results will be beneficial for all the RoboCup@Home community.

Robot's Hardware and Software Description

The RoboCup@Home Social Standard Platform competition will be run with the standard platform Softbank Robotics Pepper. A (preliminary) list of software components that we plan to use in the team is shortly summarized here and described in more details in the website (which includes also link to publicly available releases).

  • Navigation, localization and mapping: integration between thin navigation, thin mapping developed within the COACHES project and strands navigation from STRANDS project.
  • People detection and tracking: strands perception people developed in STRANDS.
  • Speech understanding: LU4R A Spoken Language Understanding Chain for HRI. Software component developed in previous projects at Sapienza and used in COACHES.
  • Speech generation: off-the-shelf component.
  • Multi-Modal Interaction Manager: MODIM package developed in COACHES integrated with STRANDS HRI package.
  • Basic actions: implementation using ROS actionlib formalism
  • Plan execution: Petri Net Plans library.
  • Planning: ROSPlan and MDP-PNP developed in [1].

References

  1. L. Iocchi, L. Jeanpierre, M. T. Lázaro, and A.-I. Mouaddib. A practical framework for robust decision-theoretic planning and execution for service robots. In Int. Conf. on Automated Planning and Scheduling (ICAPS), pages 486–494, London, UK, June 12-17 2016.
  2. M. Hanheide et al. Robot task planning and explanation in open and uncertain worlds. Artificial Intelligence, 2015.
  3. F. M. Carlucci, L. Nardi, L. Iocchi, and D. Nardi. Explicit representation of social norms for social robots. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2015.
  4. L. Iocchi, M. T. Lázaro, L. Jeanpierre, and A.-I. Mouaddib. Personalized shortterm multi-modal interaction for social robots assisting users in shopping malls. In Int. Conf. on Social Robotics, Paris, France, Oct 26-30 2015.
  5. M. T. Lázaro, F. Montesanti, A. Vanzo, and L. Iocchi. High-level formalization and execution of human-robot multi-modal interactions. In Under review.
  6. C. Dondrup, N. Bellotto, M. Hanheide, K. Eder, and U. Leonards. A computational model of human-robot spatial interactions based on a qualitative trajectory calculus. Robotics, 4(1):63, 2015.
  7. Manja Lohse, Marc Hanheide, Karola Pitsch, Katharina J Rohlfing, and Gerhard Sagerer. Improving HRI design by applying Systemic Interaction Analysis (SInA). Interaction Studies, 10(3):298–323, dec 2009.
  8. J. Pulido Fentanes, T. Krajnik, M. Hanheide, and T. Duckett. Persistent localization and life-long mapping in changing environments using the frequency map enhancement. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Daejeon, Korea, Oct. 9 - 14 2016.
  9. J. Pulido Fentanes, B. Lacerda, T. Krajník, N. Hawes, and M. Hanheide. Now or later? predicting and maximising success of navigation actions from long-term experience. In Int. Conf. on Robotics and Automation (ICRA), 2015.
  10. Marc Hanheide, Denise Hebesberger, and Tomas Krajnik. The When, Where, and How: An Adaptive Robotic Info-Terminal for Care Home Residents – A long-term Study. In Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), Vienna, 2017. ACM/IEEE.