View Original PDF

LiU@HomeWreckers 2018 Team Description Paper

Sofia Thunberg, Mattias Tiger, Fredrik Löfgren

Department of Computer Science and Information Science Linköping University, 581 83 Linköping, Sweden

http://www.ida.liu.se/divisions/hcs/coin/robocupathome/

Abstract This is the application for the RoboCup@Home 2018 Social Standard Platform League from the newly formed LiU@HomeWreckers team. The main research line is in human robot interaction with the goal for the Pepper robot to engage in experiments in real Swedish homes. This research is important for the social robotics community and triggers many other research areas, such as navigation and natural language processing. The result from the upcoming year will be used to increase social acceptance of robots in the society to be able to work towards humanoid robots living in homes of elderly and people with cognitive disabilities.

Introduction

We are a newly formed RoboCup@Home SSPL team with strong ties to human computer systems research, and in particular human robot interaction (HRI) research. The competition is an opportunity for us to engage with the wider social robotics community to share and leverage knowledge and source code in a friendly atmosphere.

One of the central concerns in human robot interaction (HRI) research is how to make interaction with robots socially acceptable and comfortable to humans [3]. One of the challenges related to this issue is to find a good balance and consistency between robot behavior and morphology. According to one classification, robot morphologies can be anthropomorphic (human-like), zoomorphic (animal-like), caricatured and/or functional [5]. Exploiting people's tendencies to anthropomorphize – to ascribe human-like characteristics to things, such as animals or inanimate things – has been suggested as a means of facilitating interaction between humans and robots [4]. In contrast to this suggestion, it has also been shown that highly human-like robots can cause feelings of discomfort or "uncanniness" in people [10, 11]. Hence, robot morphology affects people's acceptance of robots as interactive partners. Walters et. al. (2006) argues that robots that are aware of their social space increase the social acceptance by humans. They found that the participants preferred a robot approaching from the left or the right side and that they disliked a frontal approach [17]. Pepper (in an experimental comparison to the Aldebaran NAO robot) has been found to have a lower degree of human social acceptance toward the robot, with a significant lower result of being likeable, intelligent, safe and lifelike [15, 16]. This result can be explained with that Pepper acts more unpredictable than NAO (with life mode on) and that Pepper is a larger and more human-like robot and can be perceived as more uncanny.

When the participants in a experiment situation think that they are talking to a computer, though they in fact are not, and the conversation is instead mediated by a human operator (the so-called wizard), it is called a Wizard of Oz-design (WoZ) [2]. There are several reasons for wanting to conduct WoZ experiments. One is that computers are rigid and that people are flexible. In an experiment when the participant interact with a computer, the situation might change quickly in an unexpected way. If the experiment relies on a static system that cannot change, the situation will not seem natural to the participant, and therefore it is better with a wizard that can adjust.

WoZ is a common employed technique in HRI, where the wizard are controlling the robot on a number of things, such as the robots movements, navigation, speech and gestures [14]. Researchers mainly uses WoZ in HRI because robots are not sufficiently advanced to interact autonomously with people in socially appropriate ways, and this method make it possible for the researcher to envision what future interaction could be like. Riek say that one methodological concern regarding WoZ is that one can argue that it is not really human-robot interaction so much as human-human interaction that is mediated through a robot. Another concern is the ethical problems that arise when the participant cannot tell whom they are interacting with, the robot or the test leader.

With all the stated issues regarding WoZ we are suggesting implementation with more autonomous features, such as:

  • Face detection and recognition.
  • Object detection and recognition.
  • Gesture detection and recognition.
  • Navigation, localization and mapping.
  • Speech recognition and generation.
  • Manipulation.

These features would make it possible for a more natural robot interaction instead of a natural wizard interaction. This will have an impact on the research in natural environments, like in a home, a classroom or a retirement facility.

The long term goal with our participation will be to increase the social acceptance for Pepper in real world environments, to be able to do research toward elderly and people with cognitive disabilities.

During the coming years we will work on building a competitive software stack for Pepper which enable us to undertake interesting new research within the fields of HRI and artificial intelligence.

Team Background

The team consist of members from two research divisions under the Department of Computer and Information Science at Linköping University; the Human-Centered Systems (HCS) and the Artificial Intelligence and Integrated Computer Systems (AIICS).

In the HCS division, the areas Human-Centered Computing, Cognitive Science and Information Systems is included.

In the area Human-Centered Computing, we do research and teaching on human-computer interaction, multi-modal interfaces including natural language and augmented reality, interaction and service design, collaborative and social computing, as well as ubiquitous and mobile computing, and accessibility.

In the field of Cognitive Science, we do research and teaching on distributed and situated cognition, cognitive ethnography, learning technologies, and design cognition. We also do research on artificial intelligence including knowledge representation, machine learning, and natural language processing. In cognitive systems we cover human factors, cognitive ergonomics, man-machine interaction as well as command and control, emergency and disaster management, safety management, and resilience engineering.

Finally Information Systems, the research and teaching focuses on decision support systems, data mining, and the world wide web including semantic web, ontologies, description languages, searching, applications and services. We also have research on information retrieval, such as document representation, personalization and search interfaces, retrieval models, ranking, retrieval tasks and evaluation of retrieval results.

For more information on research at HCS see: http://www.ida.liu.se/ divisions/hcs/index.en.shtml

AIICS has a long history of research in artificial intelligence and its application to intelligent artifacts. Intelligent artifacts are defined as man-made physical systems containing computational equipment and software that provide them with capabilities for receiving and comprehending sensory data, for reasoning and for rational action in their environment. Examples of such artifacts range from PDAs and software agents to ground and aerial robots. An equally important focus is the development of integrated systems which include hardware, software, sensors and human users. AIICS has focused mainly on unmanned aerial vehicles but the techniques and technologies developed can also be applied to humanoid robots. Our research includes for example logic-based spatio-temporal reasoning over streaming data, automated task and motion planning, task allocation in multi agent systems, and localization and navigation of robots.

For more information on research at AIICS see: http://www.ida.liu.se/ divisions/aiics/

Some team members have experience competing in the RoboCup soccer SPL league using Nao robots. The Linköping team Linköping Humanoids, sometimes called LiU Humanoids, has competed for the last three years. The @Home SSPL provide us with an opportunity to leverage experience from the soccer SPL for the interesting and challenging tasks found in social robotics.

Re-usability and applications in the real world

We base our software stack on the Robot Operating System (ROS) [12] which is a well known middleware in the robotics community. It is widely used by research groups and companies when doing research into AI-Robotics, allowing various groups to create standardised packages for various tasks and algorithms, and to share these packages with the wider community. We are using the latest stable version of ROS, ROS Kinetic. By using ROS and by following the ROS guidelines it is easier for us and others to re-use our components and software stack. We are dedicated to the idea of giving back to the community and we will consequently release our source code as open source after the competition if accepted.

The Pepper robot platform is used by Linköping University and its representatives to promote robotics and artificial intelligence in the society at large and in the corporate sphere. We make frequent use of Pepper (and Nao) in various exhibitions, expos, demonstrations, school-visits, seminars and so on. Software developed will be used both in settings where the robot will interact with the public as well as in future HRI research.

Research Focus

AIICS is interested in the integration aspect of the complex software required for autonomous humanoid robots. We are further interested in artificial intelligence research building upon a sophisticated robotic software stack capable of operating in unstructured human environments under the presence of humans in a safe way.

HCS is interested in the interaction aspect between humans and robots, and especially how the robots behaviour is perceived by the humans. We are further interested in learning how to design robots, in terms of appearance, body language, behavior and dialogues, as to be as accommodating to humans as possible during human robot interactions in the future.

Approaches

We use several standardized components from the ROS community and elsewhere. We use for example Google cloud services for speech-to-text and text translation. To do more advanced NLP we use the open source project spaCy [6]. For face recognition we use OpenFace [1] and for arm control we use MoveIt [8].

Object detection and classification using the YOLO Detector with Darknet.
Object detection and classification using the YOLO Detector with Darknet.

Object Detection

Human and object detection is an important capability and we are utilizing modern Deep Learning based approaches such as the YOLO Detector [13] to get good performance. Figure 1 show an example of the YOLO detector output of our mock-up living room. We are further working on integrating state of the art pixel-based semantic segmentation [9].

2D and 3D mapping

A core problem to most tasks is exploration and navigation in a previously unseen environment. In order to facilitate such capabilities it is necessary to perform sufficiently good mapping of the environment and localization within the environment using said mapping. We are evaluating several 2D and 3D mapping approaches, using both the laser sensors and the RGB-D sensor. We are investigating and comparing Google Cartagropher [7] and the ROS package gmapping when using both the RGB-D sensor and the laser sensors. Figure 2 show an example of a 3D map built online. We are among other things looking into how to make these approaches work robustly for Pepper, where we for example compensate for the eye-lenses in front of the RGB-D sensor.

Top: 3D voxel map of the corridor seen at the bottom.
Top: 3D voxel map of the corridor seen at the bottom.

Conclusions

LiU@HomeWreckers is a newly formed team which are interested in highly functional robots operating in unstructured human environments in the presence of humans. We are interested in how to integrate sophisticated components and build missing components to be competitive in the RoboCup@Home SSPL. We are further interested in using the resulting platform and software stack in order to do interesting human robot interaction research as well as artificial intelligence research. We have experience with integrating complex robotic systems, work with depth data for mapping and navigation, and human robot interaction. We consequently hope that we can contribute to the @Home SSPL and to be allowed to grow into a strong and competitive team.

Robot's Hardware Description

The Pepper robot is a humanoid robot by Aldebaran Robotics and SoftBank designed as a research robot for schools, colleges and universities to teach programming and conduct research into human-robot interactions.

Specifications are as follows:

  • Height: 1200 mm.
  • Weight: 28 kg.
  • Gyro sensor.
  • 10.1-inch touch display on the robots chest.
  • 4 microphones.
  • 2 RGB cameras.
  • 1 3D sensor.
  • 3 touch sensors on the robots head.
  • 2 touch sensors on the robots hands.
  • 2 sonar sensors on the robots leg.
  • 6 laser sensors.
  • 3 bumper sensors.

Also our robot use the following external services:

  • Google Cloud Speech Recognition
  • Google Cloud Translate
Pepper.
Pepper.

Robot's Software Description

For our robot we are using the following software:

  • Navigation, localization and mapping: ACML, Octomap, Cartographer from Google and gmapping.
  • Face recognition: NAOqi, HOG and OpenFace.
  • Speech recognition, processing and generating: Google Cloud Services, SpaCy and NAOqi.
  • Object recognition: YOLO Detector with Darknet.
  • Arms control and two-hand coordination: NAOqi and MoveIt.
  • Gesture recognition: OpenNI.
  • General programs: ROS Kinetic, Choregraphe 2.5.5, Python 2.7, NAOqi 2.5.5 and OpenCV 3.3.0.

References

  1. Amos, B., Ludwiczuk, B., Satyanarayanan, M. (2016). "Openface: A generalpurpose face recognition library with mobile applications," CMU-CS-16-118, CMU School of Computer Science, Tech. Rep.
  2. Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993). "Wizard of Oz Studies-Why and How". , 1–27.
  3. Dautenhahn, K. (2007). "Socially intelligent robots: dimensions of human–robot interaction". Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1480), 679-704.
  4. Duffy, B. R. (2003). "Anthropomorphism and the social robot". Robotics and Autonomous Systems., 42(3), 177-190.
  5. Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003). @A survey of socially interactive robots". Robotics and Autonomous Systems, 42(3), 143-166.
  6. Honnibal, M., Johnson, M., (2015) "An Improved Non-monotonic Transition System for Dependency Parsing", Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1373–1378
  7. Hess, W., Kohler, D., Rapp, H., and Andor, D. (2016). "Real-Time Loop Closure in 2D LIDAR SLAM", Robotics and Automation (ICRA), 2016 IEEE International Conference on. IEEE, 2016. pp. 1271–1278.
  8. Ioan A. Sucan and Sachin Chitta, "MoveIt!", [Online] Available: http://moveit. ros.org
  9. Järemo Lawin, F., Danelljan, M., Tosteberg, P., Bhat, G., Khan, F. S., Felsberg, M. (2017). "Deep Projective 3D Semantic Segmentation", 17th international Conference on Computer Analysis of Images and Patterns (CAIP).
  10. MacDorman, K. F., & Ishiguro, H. (2006). @The uncanny advantage of using androids in cognitive and social science research@. Interaction Studies, 7(3), 297- 337.
  11. Mori, M. (1970). "Bukimi no tani [The uncanny valley]", Energy, 7 (4), 33-35.
  12. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., ... & Ng, A. Y. (2009, May). "ROS: an open-source Robot Operating System". In ICRA workshop on open source software Vol. 3, No. 3.2, p. 5
  13. Redmon, J., Divvala, S. K., Girshick, R. B. (2016). "You only look once: Unified, real-time object detection". In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 779-788.
  14. Riek, L. (2012). "Wizard of Oz Studies in HRI: A Systematic Review and New Reporting Guidelines". Journal of Human-Robot Interaction, 1 (1), 119–136.
  15. Thunberg, S., (2017). "Investigating the Social Influence of Different Humanoid Robots", Bachelor thesis, Linköping University.
  16. Thunberg, S., Thellman, S., & Ziemke, T. (in press). "Don't Judge a Book by its Cover: A Study of the Social Acceptance of NAO vs. Pepper. 5th International Conference on Human-Agent Interaction". 5th International Conference on Human-Agent Interaction, Bielefeld, Germany, October 17-20.
  17. Walters, M., Dautenhahn, K., Woods, S., Koay, K., Te Boekhorst, R., & Lee, D. (2006). "Exploratory studies on social spaces between humans and a mechanicallooking robot". Connection Science, 18 (4), 429–439.