View Original PDF

SocRob@Home: Team Description Paper

Daniel Marta, Diogo Serra, Emilia Brzozowska, Guilherme Lawless, Jhielson Pimentel, Joao Azevedo, Luis Luz, Miguel Silva, Mithun Balussery, Oscar Lima, Rute Luz, Tiago Dias, Rodrigo Ventura, Pedro Lima

Institute for Systems and Robotics; Instituto Superior Tecnico; University of Lisbon, PORTUGAL

http://socrob.isr.ist.utl.pt · http://welcome.isr.tecnico.ulisboa.pt/isrobonet/

Abstract The SocRob team has been representing ISR/IST since 1998 in the world's leading scientific event on Artificial Intelligence and Robotics, RoboCup, as the application side of SocRob (Soccer Robots or Society of Robots) ISR/IST research project. The project has involved more than 40 students over these 16 years, from early MSc years to PhD students, and has reached a maturity level that enables behavior development that integrates low level robot skills such as navigation, perception and manipulation into more complex behaviors that allow the completion of certain home tasks. Until 2013, the team's participation has encompassed Simulation, 4-Legged, Middle Size and Robot Rescue Leagues in several editions of the RoboCup World Championship and various regional RoboCup events, e.g., the Portuguese, German and Dutch Opens.

I. TEAM DETAILS

• Team Name: SocRob@Home

• Project Coordinators: Prof. Pedro Lima, Prof. Rodrigo Ventura

• Team Leader: Oscar Lima

• Link to the team web site: http://socrob.isr.ist.utl.pt

• Affiliation: Institute for Systems and Robotics (ISR), Instituto Superior Tecnico (IST), University of Lisbon, Portugal.

II. INTRODUCTION AND SCIENTIFIC BACKGROUND

Given that the focus and goals of ERL Service Robots are on the same lines as of RoboCup@Home, participating in the ERL Barcelona Competition 2017 would be an ideal opportunity for our team. As a precursor to this, our team had the great chance to participate in the RoCKIn Camp 2014, organized in Rome, Italy, where we not only obtained theoretical and hands-on experience in to this new domain but also showcased our robot's capabilities that eventually led us to receive the award for "Best in Class for Manipulation". We then participated in the 'FreeBots' league in the Portuguese Robotics Open 2014. As teams are free to choose and demonstrate any robot(s) of their choice in this league, we successfully demonstrated our robot assisting its owner to receive a package (registered or unregistered mail) from a postman by attending the latter at the door. In March 2015 we participated in the RoCKIn Camp 2015, in Peccioli, where our robot was demonstrated in the intelligent home of ECHORD++ RIF at U. Pisa's Service Robotics and Ambient Assisted Living Lab, winning the RoCKIn@Home Benchmarking Award. More recently, we participated in the @home league of the RoboCup GermanOpen 2015 and 2017 event in Magdeburg, where we received the "Most Appealing Robot Award". In addition, our participation was praised by the competition organizers as above expectations for a first participation on this very challenging league.

More details regarding this participation can be obtained from our team's homepage. Furthermore, SocRob@Home benefits from an official ERL testbed (ISRoboNet@Home) which allows the team to test the robot abilities in a real apartment scenario, while recording benchmark data from a state of the art localization system from OptiTrack®.

The broader goals of SocRob include inculcating in young researchers the ability to work as part of an engineering team, to solve engineering problems of diverse types (from hardware to software, including wireless communications, navigation, control, electronics, computer engineering, software engineering), to integrate contributions from modern information and communication technologies (e.g., networked robot systems require a mobile wireless network with robots, off-board computers, external sensors) and to ensure a background that opens doors for future bright multi-faceted engineers or engineering researchers.

The rest of the paper is organized as follows. In Section III we describe our research objectives and the goals we envisage through our participation in the @Home-type competitions in general. Section IV provides a detailed description of the robotic platform we intend to use in the @Home-type competitions as well as take to the ERL competition 2017. Towards the end of this paper, in Section V, we provide a summary of the most relevant achievements of our team members.

III. RESEARCH OBJECTIVES AND GOALS

Domestic robotics is a rapidly growing field of research with applications ranging from simple robotic machines for house cleaning to much smarter companion robots intended to provide care for the elderly at home. Robotic systems capable of providing such assistance to humans not only need to address the issues of sensor-fusion, decision-making and complex manipulations but should also posses a highly natural human-robot interaction skills. Based on our past research experience and skills, we intend to address the sensorfusion and decision-making problems using the concepts of distributed and networked-robotic systems (NRS). Initially, it will involve a single robot in an NRS with a network of static sensors if available in the environment. Later, this could be extended to a team of mobile robots within an NRS. We further enumerate our research-specific objectives through @Hometype participation.

  • Navigation: Most algorithms in robotics expose many parameters to configure which typically are hand-tuned. In this paper we propose a method to automatically tune the parameters of robotics algorithms. The use case is for the well known algorithm Adaptive Monte Carlo Localization (AMCL). As a result we have improved the localization accuracy of our robot by automatically tuning the localization parameters using several recorded training datasets.
  • Perception and Sensor Fusion: Our research in this domain includes vision-based robot localization, object tracking, simultaneous localization and tracking (SLOT), environment modeling, laser-based robot localization and, vision-based simultaneous localization and mapping (SLAM).

Particle filter-based (PF) methods have been the focus of our research to address most perception-related problems. Using PFs, the key issues that we have been engaged in solving includes i) fusion of noisy sensory information acquired by mobile robots where the robots themselves are uncertain about their own poses, and ii) scalability of such fusion algorithms w.r.t. the number of robots in the team as well as the number of objects being tracked.

For a domestic service robot working in a @Home-type environment, localization, mapping and object/person tracking constitute the basic requirements. In addition to this, static sensors along with mobile robots in an NRS, introduce further challenging issues for sensorfusion algorithms. Considering these, we intend to actively drive-forward our perception-related research in SocRob@Home.

• Decision Making: In prior work, we have addressed the problem of decision making for teams of autonomous robots, primarily through approaches based on the theory of Discrete Event Systems (DES), and also through decision-theoretic formalisms for multiagent systems (Partially Observable Markov Decision Processes – POMDPs). Recently, we have bridged these two modeling approaches, through the development and application of event-driven decision-theoretic frameworks. The fundamental insight of this line of research is that decision making in physical environments is typically an asynchronous, event-driven process over several levels of abstraction, based on limited or uncertain sensorial information over each level, and subject to uncertain outcomes. We have explored this approach in the ongoing MultiAgent Surveillance Systems (MAIS+S) project (ref. CMU-PT/SIA/0023/2009), where we have successfully implemented an NRS for autonomous surveillance, comprising a team of mobile mobile robots and a set of stationary cameras. The system is able to automatically detect relevant events in its operational environment, and the robot team can cooperatively decide on the appropriate response. In this context, we have also developed a suite of software tools to aid researchers in the systematic deployment of these abstract, decision-theoretic methodologies on autonomous robots (the Markov Decision Making Library).

We seek to continue our work in this topic in SocRob@Home, noting that the ability to perform decision-making under uncertainty is a fundamental requirement of any potential domestic robot: given multiple tasks, such a robot must be able to manage their priorities; establish a plan for each of them; and still be able to react, reliably, to external events. Automated dialogue systems, which we also plan to develop as part of our research effort in SocRob@Home, can also be interpreted as partially observable decision making problems.

  • Manipulation: Although we have less experience so far in this area, researchers in our team target the pick and place scenario from different sources like small and big tables, couch and floor. We have a 7 degree of freedom manipulator to accomplish those goals (detailed in Section IV). Simultaneously, we are also developing a torque control interface for the gripper and a visual servoing functionality.
  • Human-Robot Interaction: We have focused on serviced robots in office environments, addressing in particular symbiotic autonomy: robots execute tasks requested by users, while autonomously aware of their own limitations, asking humans for help the robot overcoming them. More recently we have been moving towards speech-based communication, in order to address the @home requirement of natural human-robot interaction.

IV. ROBOT DESCRIPTION (HARDWARE AND SOFTWARE)

Our robot builds upon a 4-wheeled omni-directional robot platform, shown in Fig. 1. This robot has been specifically developed for an ongoing European FP7 project: MOnarCH. In addition to various other sensors and actuators as described in the literature, it is equipped with two laser range finders (LRFs) which are used for mapping, navigation, and obstacle avoidance, an Kinect RBG-D camera, and a display with touch screen. On top of this platform, we installed additional devices, namely, a 7 DoF arm for manipulation (Robai Cyton Gamma 1500), a directional microphone for speech interaction (Røde VideoMic Pro), and an additional RGB-D camera for object detection (Asus Xtion PRO Live), recognition, and location.

The software architecture is based on ROS for middleware, while using off-the-shelf components whenever possible. This allows the team to focus on our research interests.

Fig. 1. Robot platform of SocRob@home.
Fig. 1. Robot platform of SocRob@home.

A. Navigation

For navigation we use move base from ROS with a standard occupancy grid map, obtained from off-the-shelf SLAM software. This map is used both for motion planning, using dijkstra, and localization using Adaptive Monte Carlo Localization - AMCL. Guidance and obstacle avoidance relies on a Dynamic Window Approach (DWA) algorithm.

B. Manipulation

We are using a Robai Cyton Gamma 1500, a 7-DoF manipulator, mounted on the base platform. The arm weight is about 2Kg with a payload of 500g. The drivers for ROS were re-written by the team, building on top the low-level dynamixel drivers provided by Antons Rebguns. Motion planning is performed by the MoveIt! library, also available for ROS. This library supports collision avoidance of the arm with obstacles (namely the robot body) during motion execution.

C. Interaction with users

Our platform supports two interaction modalities: (1) touch interface over a Graphical User Interface(GUI), and (2) speech synthesis and recognition. Speech interface is currently taskoriented, that is, the dialogue with the user is tailored towards the execution of a specific task.

Text-To-Speech (TTS) employs the eSpeak package, while Automatic Speech Recogniton (ASR) is based on VoCon (by Nuance), a state-of-the-art commercial solution. It is a grammar based ASR, and the grammars are created based on a previous knowledge of the lexicon of the scenarios that the robot will have to be able to understand. To improve recognition rate, a confidence-based threshold is defined according to which utterances may be discarded. Speech understanding is based on the definition of a grammar over a corpus, which spawns the possible sentences the ASR recognizes.

D. Object recognition

The object recognition module is based on the 3D recognition framework of the b-it-bots RoboCup team. Using the PCL library and the RGB-D camera on top the robot for point cloud acquisition, it comprises of two modules: detection and recognition pipeline.

The training module contains a script for collecting pointcloud data from the objects and uses a Support Vector Machine SVM for classification.

The object detection process is comprised of three main steps, horizontal plane detection, scene segmentation and clustering. Some of the features used for classification include the dimensions of the bounding box and the mean circle that fits the object.

E. People following

The people following behavior is divided in three modules: detection, tracking and following. We use both RGB and RGBD data comming from a Asus Xtion camera mounted on the robot's head.

Detection is performed using Aggregate Channel Features ACF on RGB images. The center of the bounding box is assumed to be the person and we get the correspondent 3D point in space from the depth pointcloud.

The 3D detection is done using the middle point of the previously detected bounding box and the depth information of the camera, this 3D point is then projected to the ground. Detections further away than a certain threshold are not considered. The detected person is then passed to the tracker based on Kalman Filter (constant velocity model) with Nearest Neighbour Joint Probabilistic Data Association (NNJPDA), from the bayes people tracker ROS package. In this case there is only one source of detections, so the NNJPDA is not relevant. The tracker outputs the position and velocity of the person. The actual movement of the robot is based on the position of the person and by sending goals to the navigation (move base) component of the robot with a certain distance from the target. The robot's neck continuously turns to face the person and to facilitate having the person inside the field of view of the camera.

V. COMPETENCE OF TEAM MEMBERS

The team covers a broad range of competences, from Mecatronics integration to high-level decision making, including perception and task planning. We briefly describe below the competence of each team member (in alphabetical order):

  • Daniel Marta is knowledgeable in Deep Learning and Reinforcement Learning techniques, in particular he is proficient with Tensor Flow open source library.
  • Diogo Serra is our Lab technician working at ISR who deals mainly with the robot hardware.
  • Emilia Brzozowska is a MSc student, she recently joined the team and is interested on working with manipulation, specifically on improving the grasping skills of the robot.
  • Guilherme Lawless is a PhD student at ISR working on Active SLAM to build metric map automatically and Semantic Mapping.
  • Jhielson Pimentel is a MSc graduate research assistant, working on the mbot simulator maintenance and the extensions needed.
  • Joao Azevedo is a MSc student, working on navigation and multi-criteria optimization.
  • Luis Luz is a MSc graduate research assistant, addressing the pregrasp planner (manipulation) and people following behavior.
  • Miguel Silva is a MSc student, dedicated to domain modeling for task planning.
  • Mithun Balussery is a MSc graduate research assistant, working on Speech recognition and Natural Language Understanding.
  • Oscar Lima is a PhD student in charge of leading the team, task integration, task planning, robust execution and monitoring.
  • Rute Luz is a MSc student, working on 3D mobile robot Navigation on partially observable stochastic environments.
  • Tiago Dias is a PhD student and is working on object recognition.
  • Pedro Lima, and Rodrigo Ventura are faculty which have been involved in robot competitions since the first editions of the RoboCup event. They are coordinating the whole team.

ACKNOWLEDGMENT

This work was supported by the FCT project UID/EEA/50009/2013. This work was supported by project PTDC/EEI-SII/4698/2014.

References

[1] R. V. Oscar Lima, "A case study on automatic parameter optimization of a mobile robot localization algorithm," ICARSC 2017, 2017. Available: https://github.com/oscar-lima/autom_param_optimization [2] P. U. Lima, P. Santos, R. Oliveira, A. Ahmad, and J. Santos, "Cooperative localization based on visually shared objects," RoboCup 2010: Robot Soccer World Cup XIV. Lecture Notes In Artificial Intelligence, pp. 350–361, 2011. Available: http://dl.acm.org/citation.cfm?id=1983806.1983837 [3] A. Ahmad and P. Lima, "Multi-robot cooperative spherical-object tracking in 3d space based on particle filters," Robotics and Autonomous Systems, vol. 61, no. 10, pp. 1084 – 1093, 2013. Available: http://www.sciencedirect.com/science/article/pii/S0921889013000535 [4] A. Ahmad, G. D. Tipaldi, P. Lima, and W. Burgard, "Cooperative robot localization and target tracking based on least square minimization," in Robotics and Automation, Proceedings of the 2013 IEEE International Conference on, May 2013. [5] P. Vieira and R. Ventura, "Interactive mapping using range sensor data under localization uncertainty," Journal of Automation, Mobile Robotics & Intelligent Systems, vol. 6, no. 1, pp. 47–53, 2013. [6] J. Ferreira, A. Vale, and R. Ventura, "Vehicle localization system using offboard range sensor networ," in Proceedings of IFAC Synposium on Intelligent Autonomous Vehicles (IAV-13), 2013. [7] F. Jesus and R. Ventura, "Simultaneous localization and mapping for tracked wheel robots combining monocular and stereo vision," Journal of Automation, Mobile Robotics & Intelligent Systems, vol. 6, no. 1, pp. 21–27, 2013. [8] G. Neto, H. Costelha, and P. Lima, "Topological navigation in configuration space applied to soccer robots," in RoboCup 2003: Robot Soccer World Cup VII. Springer, 2004, pp. 551–558. [9] G. Neto, "Planning, learning and control under uncertainty based on discrete event systems and reinforcement learning," Ph.D. dissertation, Instituto Superior Tecnico, Universidade Tecnica de Lisboa, 2010. [10] H. Costelha and P. Lima, "Modelling, analysis and execution of robotic tasks using Petri Nets," in IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2007, pp. 1449–1454. [11] J. Messias, M. T. J. Spaan, and P. U. Lima, "Efficient offline communication policies for factored multiagent POMDPs," in Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS '11), 2011, pp. 1917–1925. [12] ——, "GSMDPs for multi-robot sequential decision-making," in Proceedings of the 27th AAAI Conference on Artificial Intelligence, 2013. [13] ——, "Multiagent POMDPs with asynchronous execution," in Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-13) - Extended Abstract, 2013, pp. 1449– 1454. [14] J. Messias, "The mdm package: Software description and deployment guidelines," Institute for Systems and Robotics, Tech. Rep., 2013. Available: http://users.isr.ist.utl.pt/~jmessias/MDM/mdm_report.pdf [15] M. Veloso, J. Biswas, B. Coltin, S. Rosenthal, S. Brandao, T. Mericli, and R. Ventura, "Symbiotic-autonomous service robots for userrequested tasks in a multi-floor building," in Workshop on Cognitive Assistive Systems (CAS), IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-12), Vila Moura, Portugal, 2012. [16] R. Ventura, B. Coltin, and M. Veloso, "Web-based remote assistance to overcome robot perceptual limitations," in AAAI Conference on Artificial Intelligence (AAAI-13), Workshop on Intelligent Robot Systems. Bellevue, WA: AAAI, 2013. [17] J. Messias, R. Ventura, P. Lima, J. Sequeira, P. Alvito, C. Marques, and P. Carrico, "A robotic platform for edutainment activities in a pediatric hospital," in Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions, 2014. [18] A. Elfes, "Using occupancy grids for mobile robot perception and navigation," IEEE Computer, vol. 22, no. 6, pp. 46–57, 1989. [19] D. Fox, W. Burgard, and S. Thrun, "The dynamic window approach to collision avoidance," IEEE Robotics & Automation Magazine, vol. 4, no. 1, pp. 23–33, 1997. [20] O. Brock and O. Khatib, "High-speed navigation using the global dynamic window approach," in Proceedings of the IEEE International Conference onRobotics and Automation, vol. 1. IEEE, 1999, pp. 341– 346. [21] S. Thoduka, S. Pazekha, A. Moriarty, and G. K. Kraetzschmar, "Rgbd-based features for recognition of textureless objects," 2016. [22] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273–297, 1995. [23] B. Yang, J. Yan, Z. Lei, and S. Z. Li, "Aggregate channel features for multi-view face detection," in Biometrics (IJCB), 2014 IEEE International Joint Conference on. IEEE, 2014, pp. 1–8.