LyonTech, Team Description Paper RoboCup@Home 2018 Social Standard Platform League

Fabrice Jumel, Raphael Leber, Eric Lombardi, Laetitia Matignon, Jacques Saraydaryan, Olivier Simonin, Christian Wolf

CPE Lyon; INSA Lyon; UCBL Lyon 1 Univ.; CITI Lab., INRIA; LIRIS Lab., CNRS

https://robocup-lyontech.github.io/ · https://tinyurl.com/LyonTechVideos · http://www.citilab.fr/ · https://team.inria.fr/chroma · https://team.inria.fr/chroma/en/plate-formes/

Abstract LyonTech consortium is ready and motivated for Robocup@home thematics and especially for the Social Standard Platform. We provide : i) highly qualified researchers in several areas of robotics (robot navigation, robot control, computer vision); ii) a fruitful collaboration between researchers and engineers; iii) past participations in the competition; iv) the integration of a large number of highly qualified students from different engineering schools (eq. Universities); v) expertise on ROS and Naoqi frameworks.

Introduction

La Doua - LyonTech is the main technology campus of Lyon, France. This campus is home to 25000 students, 1500 researchers and 1200 PhD students, spread out over a 100 hectare area. Different organizations inside the campus gathered to create the LyonTech team in order to participate to the Robocup@home challenge. The LyonTech team members belong to three teaching entities, two research laboratories, and mainly to the Chroma research team from INRIA :

• CPE Lyon, Engineering school, former robocup team in 2013 and 2016
• INSA Lyon, Engineering school, candidate for robocup 2016 organization
• University Claude Bernard Lyon 1 and its engineering school Polytech Lyon
• CITI Lab., Centre of Innovation in Telecommunications and Integration of Service (INRIA)
• LIRIS, Vision and information system laboratory (CNRS)

As we gather to create this new team, we aim to use our synergy to contribute to the Robocup@home thematics with the Social Standard Platform.

This paper is organized as follows:

– Research and engineering interests

• Previous results and contributions to Robocup and Robocup@home
• Our solutions for the SSPL
• Conclusion and references

Research and engineering competences

The LyonTech consortium consists of highly qualified researchers in computer science (AI, vision, navigation) working in research and teaching environments together with groups of highly skilled students.

Teaching involvement

The 3 teaching entities mentioned in introduction are very active in the robotic field.

CPE Lyon is a 130 years old engineering school (chemistry and computer science diplomas) that offers a "service robotics" speciality (fourth and fifth year), based on software development for robotics (the robotic framework ROS), embedded systems, and a bit of mechatronics. The courses are based on a lot of practical work on different platforms (custom ROS plateforms, Youbot, Baxter, Naos, Peppers, Turtlebots, UAVs ...). To finish their training, students have to work on a robotic project. Robocup@home is a convenient opportunity for students to work on a challenging global project, so we decided to keep working on this Robocup@home project for the following years.

INSA Lyon is a leading engineering school belonging to the French INSA institute (National Institute of Applied Sciences). INSA Lyon boasts 23 research laboratories, more than 600 researchers and teacher-researchers, 650 PhD students and over 1,000 industrial contracts with the socio-economic world. It has been now 4 years that the Telecom Department has been offering a "Robotics & A.I." specialization during the fifth year. Students are trained to autonomous navigation (perception, mapping, planning) and multi-robot cooperation. They also carry out a project, under ROS system, with mobile robots among one Pepper humanoid, 20 Turtlebot robots and 5 Parrot Bibop UAVs. Since 2017 some of them are working on human-aware navigation with Pepper for the Robocup@home project.

Polytech Lyon is an engineering school integrated to University Claude Bernard Lyon 1. The school's IT Department offers an "agent and multi-agent systems" module that focuses on artificial intelligence, considering various learning approaches (machine learning, developmental learning) or bio-inspired methods. Specific courses in this module are dedicated to a project where students have to apply these techniques to robotics, using robotic framework ROS and a platform composed of mobile robots. Robocup@home provides a suitable and motivating setting for this project.

Research competences

Research on robotics and application are well developed and structured. Two laboratories are involved in the LyonTech team.

CITI is an academic laboratory associated with INSA Lyon and INRIA. The CITI Laboratory develops research activities bringing together computer science, networking, and digital communications to address the challenging issues related to the development of IoT, fleet of connected vehicles and robots. The Lab. houses around 100 people, who are organized in 6 teams (see http://www.citilab.fr/).

Among these teams, Chroma is an INRIA team-project dedicated to humanaware navigation and multi-robot systems (https://team.inria.fr/chroma). Jacques Saraydaryan, Fabrice Jumel, Laetitia Matignon, Christian Wolf and Olivier Simonin are members of the Chroma team (led by Prof. O. Simonin), which researchs focus on human-aware robot navigation and cooperation in dynamic environments.

LIRIS is a laboratory of the French Center of National Research (CNRS) with more than 300 researchers (140 research staff) from University of Lyon working in a variety of computer science fields: Computer Vision, Machine Learning and AI, geometry and modeling, data science, services, distributed systems, security, simulation, virtuality, computational sciences, interactions and cognition. The members involved in the LyonTech project are part of two different work groups, namely the Computer Vision Group (Imagine) and the Multi Agent Systems Group (SMA).

The Robocup@home challenge is an opportunity for team members to work on their specialties (image analysis, navigation, robot fleet management). It helps them to define use cases to drive research focus. For example the cases of robot waiter and tour guide robot are directly considered in our study benchmarks for navigation.

Previous results and contributions to Robocup and Robocup@home

LyonTech is composed by former members of CPE Lyon team and by former candidates for Robocup organization.

• 'Lyon CPE" team : 3rd place at Robocup@work, Joao Pessoa, Brazil, 2013
• "CPE Robot Forum" team : 15th place at Robocup@home OPL Leipzig, Germany, 2016
• Lyon city and INSA candidated for the organization of the Robocup, in 2016 (co-led by O. Simonin from Chroma/CITI team).
• Fabrice Jumel (CPE Lyon/CITI) is a Robocup@home evangelist for France, linked with the application of Bordeaux for Robocup 2020 and the future French Robocup regional committee. He is TC Robocup@home for 2017-2018. Organizer of a French @home open (first edition 24, 25th January 2018 in Lyon). He was OC for Robocup@home SSPL in Nagoya and Robocup@home LARC in Recife.

Our solutions for the SSPL

The architecture of LyonTech's embedded AI software is shown in Figure 3. It contains modules which have been developed in different research groups of the consortium, completed by off-the shelf modules which tackle standard tasks, as well as engineering bricks interconnecting these modules. The scientific expertise of the consortium is broad and targets the needs of the competition:

Perception Our computer vision experts bring knowledge in gesture recognition [1], activity recognition [2, 3], articulated pose estimation [4], semantic segmentation [5] and object recognition [6]. A large part of these methods are capable of running on real time and have been integrated in our platforms of mobile robots. Our combined work allows us to be aware of the objects present in a room, their locations, as well as the ongoing activities in this room.

Motion planning and Decision making Our expertise in robotics relates to motion planning in dynamic and uncertain environments, mapping, localization and decision-making for single and multi-robot systems. The work focuses on autonomous navigation in crowded environments (human-aware navigation) and in urban traffic (autonomous vehicles) for human assistance [7–9]. We also explore robot fleet cooperation for human scene observation [10], 3D environment mapping, transport and service delivery [11]. We experiment and evaluate the models with Pepper humanoids, fleets of mobile robots (cf. Figure 2) and UAVs, and two equipped/autonomous cars (see https://team.inria.fr/chroma/en/plate-formes/).

Human-Robot Interactions We have been working for years on different interactions with robots (from teleoperation to multirobots orchestration [12– 15]). Since this year, we also get strong support from the Hoomano company, which creates solutions for Human-Robot interaction in real world applications, in particular for Pepper robots.

Integration All components are integrated using the ROS middleware, which is the base system of our robot. ROS offers the ability to connect a set of programs through synchronous and asynchronous communication. Moreover, by allowing a set of heterogeneous components (probes, actuators, services) to communicate in a normalized way, processing program can be reused. The naoqi sdk, provided by softbank/aldebaran with the Pepper robot, gives a set of API that is mainly used for Robot and Human interactions (speech recognition, text to speech, robot behavior feedbacks). In order to highlight the robot activity, the Pepper tablet gives visual feedbacks (javascript framework).

We define five principal functionality blocks in our architecture (figure 3). The Robot Navigation Manager is in charge of localizing the robot and allowing dynamic navigation (obstacles avoidance). The analysis of the robot environment is performed by Object Detection and recognition modules, mainly deep neural networks developed inhouse [6] or off-the-shelf modules like YOLO 9000 [16]. Labeled object positions are provided to other blocks. All human robot interactions are managed by the Robot Human interaction block embedded in the robot. The robot also maintains a knowledge database about its environment (humans, objects and points of interest positions).

Finally, the general manager block works like an orchestrator and gives order to other blocks in order to achieve scenarios.

Conclusion

We gave an overview of the approach which will be used by the LyonTech team to target the SSPL Robocup@home competition, including the different AI modules developed in the different research groups of the consortium. Fig. 4 and the video submitted with this paper present a scenario illustrating our ongoing work.

We believe in the following strengths of the LyonTech consortium: i) highly qualified researchers in several areas of robotics which are vital for this competition (robot navigation, robot control, computer vision); ii) a fruitful collaboration between researchers and engineers; iii) past participations in the competition which allowed us to gain valuable experiences; iv) the integration of a large number of highly qualified students from different engineering schools (eq. Universities).

References

1. N. Neverova, C. Wolf, G.W. Taylor, and F. Nebout. Moddrop: adaptive multimodal gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 38(8):1692–1706, 2016.
2. F. Baradel, C. Wolf, and J. Mille. Human action recognition: Pose-based attention draws focus to hands. In ICCV Workshop on Hands in Action, 2017.
3. F. Baradel, C. Wolf, and J. Mille. Pose-conditioned spatio-temporal attention for human action recognition. Pre-print: arxiv:1703.10106, 2017.
4. N. Neverova, C. Wolf, G.W. Taylor, and F. Nebout. Hand pose estimation through weakly-supervised learning of a rich intermediate representation. To appear in Computer Vision and Image Understanding (CVIU), 2017.
5. D. Fourure, R. Emonet, E. Fromont, D. Muselet, A. Trémeau, and C. Wolf. Residual conv-deconv grid network for semantic segmentation. In British Machine Vision Conference (BMVC), 2017.
6. B. Moysset, J. Louradour, and C. Wolf. Learning to detect and localize many objects from few examples. Pre-print: arxiv:1611.05664, 2016.
7. F. Jumel, J. Saraydaryan, and O. Simonin. Mapping likelihood of encountering humans: application to path planning in crowded environment. In The European Conference on Mobile Robotics (ECMR), Proceedings of ECMR 2017, Paris, France, September 2017.
8. M. Barbier, C. Laugier, O. Simonin, and J. Ibanez-Guzman. Classification of Drivers Manoeuvre for Road Intersection Crossing with Synthetic and Real Data. In 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, United States, June 2017.
9. M. Andries, O. Simonin, and F. Charpillet. Localisation of humans, objects and robots interacting on load-sensing floors. IEEE Sensors Journal, PP(99):12, 2015.
10. L. Matignon, S. D'Alu, and O. Simonin. Multi-robot human scene observation based on hybrid metric-topological mapping. In European Conference on Mobile Robotics (ECMR), 2017.
11. J. Saraydaryan, F. Jumel, and O. Simonin. Robots delivering services to moving people : Individual vs. group patrolling strategies. In IEEE ARSO, 2015.
12. A. Gréa, J. Saraydaryan, F. Jumel, and A. Guenard. A robotic and automation services ontology, architectures logicielles pour la robotique autonome, les systèmes cyber-physiques et les systèmes auto-adaptables. In CAR, 2015.
13. L. Sevrin, N. Noury, N. Abouchi, F. Jumel, B. Massot, and J. Saraydaryan. Preliminary results on algorithms for multi-kinect trajectory fusion in a living lab. In IRBM, 2015.
14. J. Saraydaryan, F. Jumel, and Adrien Guenard. Astro: Architecture of services toward robotic objects. In IJCSI, 2014.
15. E.Nauer A. Cordier, E. Gaillard. Man-machine collaboration to acquire adaptation knowledge for a case-based reasoning system. In ACM DL, editor, WWW 2012 – SWCS'12 Workshop, pages 1113–1120, Lyon, France, April 2012. ACM.
16. J. Redmon and A. Farhadi. Yolo9000: Better, faster, stronger. In CVPR, 2017.