UTS Unleashed — Athome Social 2018

Mary-Anne Williams, Sammy Pfeiffer, Jonathan Vitale, Meg Tonkin, Suman Ojha, Richard Billingsley, Sidra Alam, Le Kang, Siva Leela Krishna Chand Gudi, Jesse Clark, Benjamin Johnston, Sarita Herse, Daniel Ebrahimian

Innovation and Enterprise Research Lab, Centre of Artificial Intelligence, UTS

http://utsunleashed.webnode.com

Abstract UTS Unleashed! were the only Australian team that qualified for 2017 RoboCup@Home Social SPL. It won the Human-Robot Interface Award and took second place in the competition. UTS Unleashed! has a strong track record of contribution at RoboCup since it joined in 2003: the team developed the first dodge and dribble behaviours in the SPL Soccer league, and won the Scientific Challenges in the SPL Soccer league in 2004, and were runners up in 2004 and 2008. The key focus of the UTS social robotics team is human-robot interaction, knowledge representation, cognitive architectures, emotional and social intelligence, decision making behaviour, software engineering, legal and ethical implications of social robots. Our work is foundational and pragmatic. We aim to develop breakthrough theory and translate into innovative practical methods, and feed our results back into theory development.

1 Introduction

The Innovation and Enterprise Research Lab (Magic Lab) at UTS is Australia's leader in Social Robotics and has Australia's only PR2 robot. We work with industry partners to develop socially intelligent robots. We investigate and develop tools to make robots socially acceptable, highly engaging and able to efficiently co-operate with humans in the home, work place and public environments [1–3].

Our interest in competing at RoboCup@Home Social SPL is to use the focus and effort required to develop a competitive Human-Robot Interaction (HRI) system that will help to build momentum and intensity towards scientific advances in socially-aware service-robotics. As a team of software-oriented roboticists we see significant value in the opportunity to use a Standard Platform like Pepper together with Open Source software to leverage and enrich the work of others in the RoboCup community. It provides a unique opportunity for our team to contribute to a major international scientific effort that has tremendous knowledge sharing and scaling benefits.

As Australia's leading social robotics group we are well positioned to disseminate scientific advances back into 'real word' applications [4–6]. We have a significant research partnership with Australia's largest bank and we collaborate with other Australian major commercial partners, such as the largest diversified property group with interests in large shopping centres and retirement villages, and one of the world's leading commercial airlines. These motivated research partners help us apply our research to real-world situations. This affords an extraordinary opportunity to test ideas, insights and prototypes in situations beyond our research laboratory and the RoboCup@Home test environment [7–9]. For this reason, RoboCup@Home Social SPL is an outstanding test-bed for our social robot applications, potentially leading to high impact research outcomes for the international robotics community.

2 Background and Main Research Contributions

The main research focus of our team is the development of easy to use, highly integrable and intuitive tools shaping social robot autonomous and intelligent behaviour for human-robot interaction, collaboration and engagement. Particular focus is given to shape social intelligence in robots, so as to allow them to safely co-exist and interact with people in human-centred environments. Hence, we aim to provide theoretical and practical outcomes that can be used by the robotics research community to efficiently orchestrate the capabilities of the robot platform (vision, speech recognition, human-robot interaction, display of emotions) drawing inspiration from cognitive and biological studies.

Our research lab has a strong track record of contributions in social robotics field. Specifically, we focus on the following key areas:

Social robot software cognitive architectures [10–13];
Web technologies for Human-Robot Interaction;
Design of social robotics commercial applications and Human-Robot Interaction experimentation [6–9];
Models of emotion[13, 14]
Legal and ethical implications of social robots[4]
Policy and governance of social robots in society

From our first team in 2003 we have focused on body and mind designs, and contributed highly innovative approaches and algorithms that have been subsequently adopted by all of the top teams. In the RoboCup 2010 Soccer Standard Platform League, we explored an experimental cognitive architecture inspired by psychological models of attention, the Attention Self-MOdifying (ASMO) Cognitive Architecture [10]. This architecture resulted from more than 10 years of continuous research in cognitive science and produced a system composed of self-contained modules decomposed by behaviour (e.g., ball chasing, kicking, defending) rather than the more typical decomposition into functionality (e.g., locomotion, localisation, planning). We continued to build upon this work at subsequent RoboCup competitions (Soccer SPL 2011 and 3D Simulation League 2011–2013 as part of Karachi Koalas Team).

Our cognitive architecture is an attempt to fill the gaps prevalent in presently available robot architectures, such as: their inability to autonomously manage the resources of the robot, their low integrability with scripts from different communities (computer vision, audio processing, navigation, etc.), their low extensibility and the difficulty adapting their components throughout experience, and their unsuitability in managing multiple goal at a time with reliable and safety mechanisms for robots' co-existence in human spaces [10, 15].

ASMO intelligent behaviours are designed to solve open and complex tasks and result from the emergence of constituent processes, rather than from careful top-down control engineering. ASMO has attention, emotion and learning mechanisms that are inspired by human intelligence. It treats each behaviour as a concurrent, independent and self-governed black box process that competes for the robot's attention to perform actions. Every process has local knowledge of the system and cannot directly control other processes, although their execution can influence the evolution of the realised dynamical system. The attention mechanism is used to mediate the competition among processes, which correspond to the set of potential actions. At every time step, each process can demand attention by providing an attention value, computed on the bases of the available environmental information. These attention values are adaptable throughout learning and experience in order to improve robots' performances. The emotion mechanism can also be used to bias the attention demanded by the processes. If the action to be triggered by the process requires using resources already demanded by others processes (e.g. moving the same arm of the robot or rotating its head in different directions), this set of processes enters into a winner-takes-all competition. Thus, the attention values are compared and the process with higher attention can proceed with triggering the action, whereas the losers will be inhibited from triggering their actions.

2.2 Web Technologies for Human-Robot Interaction

We created a Web-based application for facilitating Human-Robot Interaction for both robotic software developers and end-users. The application is named Pepper Monitor. Although it was designed for Pepper robot initially, it works for any ROS-enabled robot platform that is equipped with a tablet. This feature is particularly aligned with the RoboCup reusability priority.

Pepper Monitor is written in JavaScript as a Single Page Application (SPA). It is lightweight and does not require additional software to be installed. It can communicate with ROS server via Rosbridge [16], subscribing and publishing topics with simple std msgs/String message type. Developers can define a display content which may contain media (emoji expression, image or camera stream) or/and dialog (text, input, buttons or even HTML snippet) in JSON format, then convert it as String and send from ROS server to Web client side.

For end-users, this application provides an additional and efficient way other than speech for communication with Pepper robot. For example, Pepper Monitor can display emoji expression to help human users understand the feeling of the Pepper robot, as it cannot directly express its emotion via facial expression.

For developers, they can send debug messages to the tablet and monitor the status of their programs during test. Also, they are able to design HTML forms to gather human users' input, so that interactive robotic software can be fast prototyped.

This application helped us win the best Human-Robot Interface Award at RoboCup 2017. We are working on improvements that will result in better and richer Human-Robot Interaction through more Web technologies for RoboCup 2018.

Our lab is actively involved in the design of robotics commercial applications to situate in public spaces and interacting with humans. For the design of these applications we use a design methodology unifying 2 practices for designing the User Experience (UX) of HRI. i) Lean UX [17] and ii) Agile Science [18].

We successfully used this methodology to create a UX design process for HRI that both assists in creating a viable application and an enhanced user experience in several domains, such as banks, shopping centres and airports [8, 7, 9]. In addition, our suggested UX design process have been used to easily design experiments to test psychological and sociological effects of HRI for the considered applications.

In this perspective, RoboCup@Home is a significant opportunity to showcase our robotics commercial applications, to gather feedback from robotics community and to test the HRI of the designed systems.

A new agile next-generation cognitive architecture, BUNJI, is being developed for RoboCup 2018 based on the insights gathered during our experience at RoboCup 2017. BUNJI means friend in several Australian Aboriginal languages.

In order to allow a range of high-level reasoning approaches a common modular low level layer is needed. We call this layer, Skills (saying a sentence, for example). Composing one or more skills creates a Capability (e.g., answering a question).

These Skills and Capabilities can be orchestrated to create plans to achieve tasks, from subtasks such as a RoboCup test (which itself can become a Capability) to a full competition test.

Capabilities are internally represented as SMACH State Machines[19] so we can take advantage of their properties and pre-existing tools.

In order to move away from the simple state machine based approach, which plagues the league we make use of a whiteboard of common knowledge called Beliefs which our reasoning engine uses in conjunction with the Capabilities of the robot to dynamically create plans to solve the RoboCup tests as if they were General Purpose Service Robot tasks.

These plans can be represented in what we call Plan file format, which can be used to be read by humans and also to be debugged with our visual interfaces. Plans can also be considered Behaviours.

Initially, easy tests may be graphically solved by using our block-like interface, once tests require a more cognitive approach other engines can be used.

We use the following external sources of software:

The Robot Operating System ROS a collection of open source tools, libraries, and conventions for developing robot behaviour across a wide variety of robotic platforms.
Machine learning algorithms from Dlib a collection of open source machine learning algorithms that can be in a wide range of domains including robotics, embedded devices, mobile phones, and large high performance computing environments.
Keras Python API for enabling Deep Learning models on Pepper Robots.

Although our architecture supports external and cloud services we do not intend to use them at competition many due to the network traffic which typically interferes. Instead we prefer to run all behaviours on-board the robot.

4 Reusability for Other Research Groups

Our research lab has placed public dissemination and re-usability as a priority in its research efforts. New projects are, by default, made publicly available.

Our research lab Github profile is growing rapidly hosting 28 repositories, in addition to the 230 repositories associated with the individual Github profiles of lab's members.

We currently plan to continue our open position on code release as part of the RoboCup@Home campaign and will develop and release stable and documented code around the following themes:

A rapid experimentation and development framework based on Python, ROS and a range of web toolkits.
Cognitive architectures for intelligent social and socially-motivated behaviour.
Behavioural and gestural control system that combine web interfaces with intelligent behaviours.
Visual systems optimised for fluent social behaviour.
Visual tools for easy debugging of complex behaviours.

As the robot platform Pepper is standardised so the re-usability is assured. Also the ROS software components can be shared with other robots.

5 Applicability of the Approach in the Real World

Our research is conducted with a direct and immediate focus on real-world applications. We are currently conducting research with Australia's largest bank and with other major commercial partners. This research is intended to have commercial applications and affords us a venue to test research in real world applications and to translate research into societal benefits. Our partnership enables us to do this without impacting our ability to publish original research and make our source code available to other research groups around the world.

Also Robocup@Home focuses in real life home environment situations so every skill developed for the contest has a direct application in the real world.

6 Conclusions and Future Work

Our team has more than 15 years experience in robotics research and development, with a particular focus on social robots co-existing in human-centric spaces and co-operating with people. We have participated at RoboCup since 2003 and the recognise that @Home League offers unparalleled opportunity to contribute and to test our social robot applications.

In addition, Sammy Pfeiffer is an active member of the Technical Committee this year. He is looking forward to working with committee members and team leaders to improve the league and make it progressively more sociable.

Our team's aim is to include focus on the social and HRI aspects in the upcoming competition for this league, so that RoboCup@Home can positively enrich specific areas of robotics challenges that it was designed to meet.

Future work will include research oriented to unify our previous cognitive architecture, ASMO, with the novel skill-based oriented system, BUNJI, to allow not only agile development and continuous integration, but also a more humanlike approach enabling robots to take decision and reactively changing plans to solve unexpected situations.

References

Mary-Anne Williams, William Judge, Jonathan Vitale, Meg Tonkin, Benjamin Johnston, and Amit Kumar Pandey. 1st workshop on human-robot engagement in the home, workplace and public space. In International Joint Conference on Artificial Intelligence, 2017.
Hugo Romat, Mary-Anne Williams, Xun Wang, Benjamin Johnston, and Henry Bard. Natural human-robot interaction using social cues. In The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 503–504. IEEE Press, 2016.
John-John Cabibihan, Mary-Anne Williams, and Reid G Simmons. When robots engage humans. IJ Social Robotics, 6(3):311–313, 2014.
Harry Surden and Mary-Anne Williams. Technological opacity, predictability, and self-driving cars. Cardozo Law Review, 38(1):121–182, 2016.
Nima Ramezani and Mary-Anne Williams. Smooth robot motion with an optimal redundancy resolution for pr2 robot based on an analytic inverse kinematic solution. In Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, pages 338–345. IEEE, 2015.
Shaukat Abidi, Massimo Piccardi, and Mary-Anne Williams. Static action recognition by efficient greedy inference. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1–8. IEEE, 2016.
Meg Tonkin, Jonathan Vitale, Suman Ojha, Mary-Anne Williams, Paul Fuller, and William Judge. Would you like to sample? Robot engagement in a shopping centre. In The 26th IEEE International Symposium on Robot and Human Interactive Communication, August 2017.
Jonathan Vitale, Meg Tonkin, Xun Wang, Suman Ojha, and Mary-Anne Williams. Privacy by design in machine learning data collection: Ux experimentation in hri setting. In AAAI Spring Symposia 2017, Designing the User Experience of Machine Learning Systems, submitted.
Meg Tonkin, Jonathan Vitale, Suman Ojha, Jesse Clark, Sammy Pfeiffer, William Judge, Xun Wang, and Mary-Anne Williams. Embodiment, privacy and social robots: May i remember you? In The 9th International Conference on Social Robotics, November 2017.
Rony Novianto. Flexible attention-based cognitive architecture for robots. PhD thesis, Faculty of Engineering and Information Technology, 2014.
Jonathan Vitale, Mary-Anne Williams, Benjamin Johnston, and Giuseppe Boccignone. Affective facial expression processing via simulation: A probabilistic model. Biologically Inspired Cognitive Architectures, 10:30 – 41, 2014.
Mary-Anne Williams. Decision-theoretic human-robot interaction: Designing reasonable and rational robot behavior. In International Conference on Social Robotics, pages 72–82. Springer, 2016.
Suman Ojha, Jonathan Vitale, and Williams Mary-Anne. A domain-independent approach of cognitive appraisal augmented by higher cognitive layer of ethical reasoning. In Annual Meeting of the Cognitive Science Society, 2017.
Rony Novianto and Mary-Anne Williams. Emotion in robot decision making. Advances in Intelligent Systems and Computing, 10(Vol. 447):221 – 232, 2017.
Rony Novianto and Mary-Anne Williams. The role of attention in robot selfawareness. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication, pages 1047–1053. IEEE, 2009.
Christopher Crick, Graylin Jay, Sarah Osentoski, Benjamin Pitzer, and Odest Chadwicke Jenkins. Rosbridge: Ros for non-ros users. In Robotics Research, pages 493–504. Springer, 2017.
Jeff Gothelf. Lean UX: Applying lean principles to improve user experience. " O'Reilly Media, Inc.", 2013.
Eric B Hekler, Predrag Klasnja, William T Riley, Matthew P Buman, Jennifer Huberty, Daniel E Rivera, and Cesar A Martin. Agile science: creating useful products for behavior change in the real world. Translational behavioral medicine, 6(2):317–328, 2016.
Jonathan Bohren and Steve Cousins. The smach high-level executive [ros news]. IEEE Robotics & Automation Magazine, 17(4):18–20, 2010.

UTS Unleashed! 2018 RoboCup@Home Social SPL