UTS Unleashed! for RoboCup 2017 @Home SPL

Mary-Anne Williams, Sammy Pfeiffer, Jonathan Vitale, Meg Tonkin, Suman Ojha, Richard Billingsley, Sidra Alam, Le Kang, Siva Leela Krishna Chand Gudi, Jesse Clark, Xun Wang, Benjamin Johnston

Innovation and Enterprise Research Lab, Centre of Artificial Intelligence, UTS

http://utsunleashed.webnode.com · http://themagiclab.org · http://github.com/uts-magic-lab/ · https://github.com/uts-magic-lab/Magiks · https://github.com/beta-robots/pipol_tracker/wiki

Abstract UTS Unleashed! has a strong track record of contribution at RoboCup since 2003. The key focus of the team is human-robot interaction, knowledge representation, cognitive architectures, social intelligence, decision making behaviour and software engineering. Our work is foundational and pragmatic. We are able to develop breakthrough theory and translate into innovative practical methods.

1 Introduction

The Innovation and Enterprise Research Lab (Magic Lab) at UTS is Australia's leader in Social Robotics and has Australia's only PR2 robot. We work with industry partners to develop socially intelligent robots. We investigate and develop tools to make robots socially acceptable, highly engaging and able to efficiently co-operate with humans in the home, work place and public environments [1–3].

Our interest in competing at RoboCup@Home SPL is to use the focus and effort required to develop a competitive HRI system that will help to build momentum and intensity towards scientific advances in socially-aware servicerobotics. As a team of software-oriented roboticists we see significant value in the opportunity to use a Standard Platform like Pepper to leverage and enrich the work of others in the RoboCup community. It provides a unique opportunity for our team to contribute to a major international scientific effort that has tremendous knowledge sharing and scaling benefits.

As Australia's leading social robotics group we are well positioned to disseminate scientific advances back into 'real word' applications [4–6]. We have a significant research partnership with Australia's largest bank and with Australia's largest diversified property group with interests in large shopping centres and retirement villages. These motivated partners will help apply the research to real world situations and also affords an extraordinary venue to test ideas, insights and prototypes in situations beyond the RoboCup@Home test environment [7]. For this reason, RoboCup@Home SPL is an outstanding testbed for our social robot applications, potentially leading to high impact research outcomes for the international robotic community.

2 Background

UTS has a strong track record of research in social robotics and competition at RoboCup that is focused on experimental software architectures based on cognitive systems [8–10].

From our first team in 2003 we have focused on body and mind designs, and contributed highly innovative approaches and algorithms that have been subsequently adopted by all of the top teams. In the RoboCup 2010 Soccer Standard Platform League, we explored an experimental cognitive architecture inspired by psychological models of attention [9]. This architecture resulted from more than 10 years of research in cognitive science and produced a system composed of self-contained modules decomposed by behaviour (e.g., ball chasing, kicking, defending) rather than the more typical decomposition into functionality (e.g., locomotion, localisation, planning). We continued to build upon this work at subsequent RoboCup competitions (Soccer SPL 2011 and Simulation League 2011–2013 as part of Karachi Koalas Team). Translating these insights from Soccer to the @Home domain will be an exciting research challenge for us.

Our ongoing research continues this line of experimentation with a focus on novel architectures for cognitive systems, such as: (i) Attention-inspired cognitive architectures for autonomous robots; (ii) Software frameworks for rapid application development in social robotics; (iii) Natural and dynamic interaction in social situations; and (iv) The integration of web technologies into intelligent robots.

3 Main Research Contributions and Directions

The main research focus of our team is the development of easy to use, highly integrable and intuitive tools shaping robot autonomous and intelligent behaviour for human-robot interaction, collaboration and engagement. Particular focus is given to shape social intelligence in robots, so as to allow them to safely co-exist and interact with people in human-centred environments. Hence, we aim to provide theoretical and practical outcomes that can be used by the robotics research community to efficiently orchestrate the capabilities of the robotic platform (vision, speech recognition, human-robot interactions, display of emotions) drawing inspiration by cognitive and biological studies. The full systems are available for download from our Github repository (http://github.com/uts-magic-lab/).

3.1 Attention Self-Modifying Cognitive Architecture

The Attention Self-MOdifying (ASMO) Cognitive Architecture is an attempt to fill gaps of presently available robotic architectures, such as: their inability to autonomously managing the resources of the robot, their low integrability with scripts from different communities (computer vision, audio processing, navigation, etc.), their low extensibility and hard adaptation of their components throughout experience, and their unsuitability in managing multiple goal at a time with reliable and safety mechanisms for robots' co-existence in human spaces [9, 11].

In ASMO, intelligent behaviours are designed to solve open and complex tasks and result from the emergence of constituent processes, rather than from careful top-down control engineering. ASMO has attention, emotion and learning mechanisms that are inspired by human intelligence. It treats each behaviour as a concurrent, independent and self-governed black box process that competes for the robot's attention to perform actions. Every process has a local knowledge of the system and cannot directly control other processes, although their execution can influence the evolution of the realised dynamical system.

The attention mechanism is used to mediate the competition among processes, which correspond to the set of potential actions. At every time step, each process can demand attention by providing an attention value, computed on the bases of the available environmental information. These attention values are adaptable throughout learning and experience in order to improve robots' performances. The emotion mechanism can also be used to bias the attention demanded by the processes. If the action to be triggered by the process requires using resources already demanded by others processes (e.g. moving the same arm of the robot or rotating its head in different directions), this set of processes enters into a winner-takes-all competition. Thus, the attention values are compared and the process with higher attention can proceed with triggering the action, whereas the losers will be inhibited from triggering their actions. Fig. 1 illustrate ASMO architecture.

3.2 Python based Robot Interactive Development Environment

Python based Robot Interactive Development Environment (PyRIDE) is a software that supports rapid interactive programming of robot skills and behaviours on a range of robots (NAO, Pepper, PR2 and REEM) including Robot Operating System (ROS) platforms [12].

PyRIDE abstracts and aggregates disparate low level ROS software modules, e.g. arm joint motor controllers and exposes their functionalities through a unified Python programming interface. Robot programmers are able to experiment and develop robot behaviours without dealing with specific details of accessing underlying software and hardware.

By providing a standard Python interactive interpreter, PyRIDE allows programs to be modified while they are running, an extremely important feature to facilitate prompt and reactive behaviours of the robot.

PyRIDE provides a client-server mechanism that allows remote user access of the robot functionalities (see Fig. 2), e.g. remote robot monitoring and control, access real-time robot camera image data etc. This enables multimodal human robot interactions using different devices and user interfaces. All these features are seamlessly integrated into one lightweight and portable middleware package.

3.3 Smart Interactions Manager Kit

Smart Interactions Manager Kit (SIMKit) is an event-driven scripting paradigm in Python building on top of an interpreter describing and executing basic motor actions and capabilities (e.g. video, audio, monitor interactions) of the robot (i.e. in our case PyRIDE). SIMKit assists with the design of complex interactions of a robot. It allows behaviours to be stored as JSON formatted scripts and to be retrieved later by using appropriate APIs.

The core objective of SIMKit library is to provide an easy to use standard for aggregating basic robot actions in complex and intelligent robot behaviours. It is possible for example to design an event-driven timeline of basic motions or capabilities of the robot, to monitor their success, manage their failure and store or retrieve sensory data, all inside a self-contained script requiring a few lines of code. Although SIMKit is still at its infancy (less than one year from its first conception), we have valued its use in the development of human-robot interactions for a robotic platform used in banks and shopping centres, thus being a natural evolution for the transaction between RoboCup Soccer League to the @Home League.

3.4 JoyRIDE Web-based Interface for Robot TouchPad

Robotic platforms are gradually incorporating monitors as a way to interact with them. More importantly Pepper (our chosen robotic platform) has one.

Our team developed JoyRIDE, a web-based interface integrated with ROS in order to display contents on the robot's monitor, such as emotional states, and retrieve inputs from users, for instance by providing customisable textboxes, keyboards and buttons. The system is currently released and tested on a REEM robot. It allows to command the monitor by sending ROS messages formatted in JSON, and it can display several layers of graphic contents. For example, in order to separate the foreground from the background or for easily managing animated transitions and transparency effects enhancing user experience.

4 Approach Planned to be Implemented on the Robot

In the past, we have used RoboCup campaigns to inspire and drive novel approaches. For the RoboCup@Home SPL 2017, we will undertake the 3 following work-packages:

We will adapt our work with ASMO and PyRIDE, to solve challenges posed by the @Home scenario. In particular, our goal is to create an attention-driven, behaviour-based social architecture. A novel feature of an attention-driven architecture is that its sub-processes are always competing to act. This means that the robot always remains in motion: it is endlessly engaging with the environment, creating a lifelike and helpful experience for people nearby.

In addition a behavioural decomposition into modules (rather than functional decomposition), makes it possible to integrate diverse reasoning mechanisms. For example, a grasping behaviour might be implemented using a range of classical planning techniques, while a social gesturing system can be trained as a reactive system based on a deep network. The use of attention to orchestrate behaviours allows for these independent behaviours to be combined into a coherent agent without requiring the uniform representations of knowledge that are required in a functional decomposition.

Develop Composable Fluent Behaviours

Unfortunately, Human-Robot interaction is typically characterised by 'robotic' interactions where the robot follows a sequence of planned state-based behaviours. The robot does not engage in social gesturing while awaiting commands, nor does it continue to gesture or attend to user commands while it is engaged in other tasks. More directly, we believe that this is due to an emphasis on designing behaviours as a sequence of planned states, rather than as a composition of independent behaviours that operate simultaneously.

After adapting our attention-based architecture to the Pepper platform, we will focus on creating natural behaviours that can be readily composed.

Refine Core-Skills and Micro-Behaviours

As far as possible, we will draw on the many resources and components already available. In this third work-package, we will focus on refining the core-skills and capabilities required for RoboCup@Home and other applications. This work will be, in the first instance, dictated by the specific rules of the RoboCup@Home SPL 2017 rule book.

5 Externally Available Components to be Implemented

Understanding externally available components as software queried from outside of the robot itself we preview the necessity of implementing:

Automatic Speech Recognition and AI conversation aids: IBM Watson; Pepper native speech recognition.
Object recognition and Computer Vision: IBM Watson; Google Vision; Object recognition kitchen; Amazon Picking Challenge.

6 Innovative Technology and Scientific Contribution

AIBO Robot Soccer Systems: PyRIDE.
NAO Robot Soccer System: PyRIDE.
Personal Robot 2 (PR2) Systems: Magiks, PyRIDE.
Customised Bear Robot: ICAPS Demo Making Humans Happy.
Pepper Robot: PyRIDE.
ROS Robot Operating System Software stack.
People tracker: pipol tracker.

7 Re-usability for Other Research Groups

Our research lab has placed public dissemination and re-usability as a priority in its research efforts. New projects are, by default, made publicly available.

Our research lab Github profile is growing rapidly hosting 28 repositories, in addition to the 230 repositories associated with the individual Github profiles of lab's members.

We will continue our open position on code release as part of the RoboCup@Home campaign and will develop and release new code around the following themes:

A rapid experimentation and development framework based on Python, ROS and a range of web toolkits, building upon our previous work with PyRIDE.
Cognitive architectures for intelligent social and socially-motivated behaviour.
Behavioural and gestural control system that combine web interfaces with intelligent behaviours.
Visual systems optimised for fluent social behaviour.

8 Applicability of the Approach in the Real World

Our research is conducted with a direct and immediate focus on real-world applications. We are currently conducting research with Australia's largest bank and with Australia's largest diversified property group. This research is intended to have commercial applications and affords us a venue to test research in real world applications and to translate research into societal benefits. Our partnership allows us to do this without impacting our ability to publish original research and make our source code available to other research groups around the world.

Also Robocup@Home focuses in real life home environment situations so every skill developed for the contest has a direct application in the real world.

9 Conclusions and future work

Our team has more than 15 years of experience in robotics, with a particular focus on social robots co-existing in human-centric spaces and co-operating with people. We have participated at RoboCup and the @Home League offers unparallelled opportunity to contribute and to test our social robotic applications.

Future work aim to create an attention-driven, behaviour-based social architecture. We will design it so to control robot platforms, while at the same time providing interpretations of the surrounding environment and social world, leading to social cognition capabilities to engage with human partners.

References

Mary-Anne Williams, Williams Judge, Jonathan Vitale, Meg Tonkin, Benjamin Johnston, and Amit Kumar Pandey. 1st workshop on human-robot engagement in the home, workplace and public space. In International Joint Conference on Artificial Intelligence. forthcoming (in preparation), 2017.
Hugo Romat, Mary-Anne Williams, Xun Wang, Benjamin Johnston, and Henry Bard. Natural human-robot interaction using social cues. In The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 503–504. IEEE Press, 2016.
John-John Cabibihan, Mary-Anne Williams, and Reid G Simmons. When robots engage humans. IJ Social Robotics, 6(3):311–313, 2014.
Harry Surden and Mary-Anne Williams. Technological opacity, predictability, and self-driving cars. Cardozo Law Review, 38(1):121–182, 2016.
Nima Ramezani and Mary-Anne Williams. Smooth robot motion with an optimal redundancy resolution for pr2 robot based on an analytic inverse kinematic solution. In Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, pages 338–345. IEEE, 2015.
Shaukat Abidi, Massimo Piccardi, and Mary-Anne Williams. Static action recognition by efficient greedy inference. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1–8. IEEE, 2016.
Jonathan Vitale, Meg Tonkin, Xun Wang, Suman Ojha, and Mary-Anne Williams. Privacy by design in machine learning data collection: Ux experimentation in hri setting. In AAAI Spring Symposia 2017, Designing the User Experience of Machine Learning Systems, submitted.
Mary-Anne Williams. Decision-theoretic human-robot interaction: Designing reasonable and rational robot behavior. In International Conference on Social Robotics, pages 72–82. Springer, 2016.
Rony Novianto. Flexible attention-based cognitive architecture for robots. PhD thesis, Faculty of Engineering and Information Technology, 2014.
Jonathan Vitale, Mary-Anne Williams, Benjamin Johnston, and Giuseppe Boccignone. Affective facial expression processing via simulation: A probabilistic model. Biologically Inspired Cognitive Architectures, 10:30 – 41, 2014.
Rony Novianto and Mary-Anne Williams. The role of attention in robot selfawareness. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication, pages 1047–1053. IEEE, 2009.
Xun Wang and Mary-Anne Williams. Pyride: An interactive development environment for pr2 robot. arXiv preprint arXiv:1605.09089, 2016.