UTS Unleashed — Athome Social 2019

Mary-Anne Williams, Benjamin Johnston, Sammy Pfeiffer, Jonathan Vitale, Jesse Clark, Le Kang, Daniel Ebrahimian, The Duc Vu, Suwen Leong, Richard Billingsley, Meg Tonkin, Sarita Herse, Sidra Alam, Siva Leela Krishna Chand Gudi, Suman Ojha

Innovation and Enterprise Research Lab (The Magic Lab), Centre of Artificial Intelligence, UTS

http://utsunleashed.webnode.com · http://themagiclab.org · http://www.computerworld.com.au/article/626400/cba-robot-chip-takes-trip-airport · https://gitlab.com/ · https://www.docker.com/ · https://docs.gitlab.com/runner/ · http://bitsandatoms.co/effective-policies-for-safety-critical-ai · http://newsroom.uts.edu.au/news/2017/05/we-are-uts · http://www.ros.org/ · http://dlib.net/ · https://keras.io/ · https://12factor.net/

Abstract UTS Unleashed! were the only Australian team that qualified for 2017 and 2018 RoboCup@Home Social SPL. It won the Human-Robot Interface Award in 2017 and took second place in the competition in both 2017 and 2018. UTS Unleashed! has a strong track record of contribution at RoboCup since its first competition in 2003: the team developed the first dodge and dribble behaviours in the SPL Soccer league, and won the Scientific Challenges in the SPL Soccer league in 2004, and were runners up in 2004 and 2008. The key focus of the UTS social robotics team is human-robot interaction, knowledge representation, cognitive architectures, emotional and social intelligence, decision making behaviour, software engineering, legal and ethical implications of social robots. Our work is foundational and pragmatic. We aim to develop breakthrough theory and translate into innovative practical methods, and feed our results back into theory development.

1 Introduction

The Innovation and Enterprise Research Lab (Magic Lab) at UTS is Australia's leader in Social Robotics and has Australia's only PR2 robot. We work with government and industry partners to develop socially intelligent robots. We investigate and develop tools to make robots socially acceptable, highly engaging and able to efficiently co-operate with humans in the home, work place and public environments [1–3].

Our interest in competing at RoboCup@Home Social SPL is to use the focus and effort required to develop a competitive Human-Robot Interaction (HRI) system that will help to build momentum and intensity towards scientific advances in socially-aware service-robotics. As a team of software-oriented roboticists we see significant value in the opportunity to use a Standard Platform like Pepper together with Open Source software to leverage and enrich the work of others in the RoboCup community. It provides a unique opportunity for our team to contribute to a major international scientific effort that has tremendous knowledge sharing and scaling benefits.

As Australia's leading social robotics group we are well positioned to disseminate scientific advances back into 'real word' applications [4–6]. We have been in a significant research partnership with Australia's largest bank leading to several research outcomes [7–11]. During this research partnership we collaborated with Australian major commercial businesses. This includes research studies in collaboration with the largest diversified property group having interests in large shopping centres and retirement villages [12], and one of the world's leading commercial airlines [13]. Currently, our research agenda is strategically considering the design, development and testing of robotics applications in public spaces in Australia by collaborating with government bodies and public hospitals. All these motivated research partners help us apply our research to real-world situations. This affords an extraordinary opportunity to test ideas, insights and prototypes in situations beyond our research laboratory and the RoboCup@Home test environment [12, 13, 7, 9, 11]. For this reason, RoboCup@Home Social SPL is an outstanding test-bed for our social robot applications, potentially leading to high impact research outcomes for the international robotics community.

2 Background and Main Research Contributions

The main research focus of our team is the development of easy to use, highly extensible and intuitive tools shaping social robot autonomous and intelligent behaviour for human-robot interaction, collaboration and engagement. Particular focus is given to shape social intelligence in robots [14], so as to allow them to safely co-exist and interact with people in human-centred environments [15, 1]. Hence, we aim to provide theoretical and practical outcomes that can be used by the robotics research community to efficiently orchestrate the capabilities of the robot platform (vision, speech recognition, human-robot interaction, display of emotions) drawing inspiration from cognitive and biological studies.

Our research lab has a strong track record of contributions in social robotics field. Specifically, we focus on the following key areas:

1. Human-Robot Engagement [1, 12];
1. Software Architecture for Continuous Integration;
1. Web technologies for Human-Robot Interaction;
1. Design of social robotics commercial applications and Human-Robot Interaction experimentation [6, 12, 7, 9, 13, 8, 10, 11];
1. Models of emotion[16–18]
1. Legal and ethical implications of social robots[4, 8]
1. Policy and governance of social robots in society

2.1 Human-Robot Engagement

Our vision consists in ensuring Pepper builds trust and enhances human wellbeing.

We employ a human-centric methodology that champions social intelligence [14] and human rights [19] with a strong focus on enhancing accessibility and human experience. We use our extensive HRI experiment outcomes [11, 17, 9, 12, 8, 1] to inform our robot behaviour designs.

Our HRI designs respect the rights of human users. For example, we ensure they are in control of how their information is collected, shared, and stored by the robot. Privacy is a basic human right and of critical importance in social robotics. This frames our research focus. We work directly with the United Nations on developing ways that ensure social robots demonstrate social intelligence [20] and respect Human rights [21, 19].

Human-Robot visual communication via our novel tablet animations increase accessibility by augmenting other modes of human-robot interaction such as sounds, speech, gesture and eye lights. They are generated from a bespoke JavaScript engine and comply with strong UTS branding guidelines.

2.2 Software Architecture for Continuous Integration

We divide our software into named packages, and store each package in a separate git repository. When new code is uploaded to the central server, it triggers a "continuous integration" (CI) pipeline for that project. The specific tasks in that pipeline vary from one project to the next, and are defined by a file in the repository and configuration variables on the server. Those tasks are run within containers on an in-house computer controlled by a daemon.

A typical CI pipeline consists of build, test, and publish stages, where each stage runs only when the previous one completes successfully. When all stages pass, the published package is uploaded to our in-house package repository. This is an special case of the "Strictly separate build and run stages" principle from "The Twelve-Factor App" [22].

The resulting package repository gives us confidence that when we install version X of a package, we are getting the code that went through the publish stage when its version was marked as X. Which in turn went through the test stage for that same commit, which has the results of those tests stored in the CI history. Ideally, we can add unit tests to the package to detect regressions in functionality and prevent new versions from being published with previously-encountered errors. When we are unable to detect regressions with testing (common for packages that depend on stateful hardware such as robots), a developer who encounters a regression can replace version X with a previous version so that their productivity is not blocked by temporary errors.

2.3 Web Technologies for Human-Robot Interaction

We created a Web-based application named Magic Tablet for facilitating Human-Robot Interaction.

Magic Tablet consists of a JavaScript frontend that runs on Pepper's chest tablet, and a Python backend that controls the frontend by sending messages through the NAOqi messaging API.

The frontend displays textual status messages at the top and bottom of the tablet screen, and an animation or image in the middle of the screen. These are typically used by interaction designers to confirm what the robot has heard, what it has said, and to characterize its state as "listening", "processing", "navigating" or so on.

We have found that abstract animations to communicate the robot's state are more appropriate than animated facial expressions, as they are displayed on the robot's chest and not its head.

We have also found that persistently displaying what the robot has just said is very helpful in noisy environments such as the RoboCup arena.

Our lab is actively involved in the design of robotics commercial applications to situate in public spaces and interacting with humans. For the design of these applications we use a design methodology unifying 2 practices for designing the User Experience (UX) of HRI. i) Lean UX [23] and ii) Agile Science [24].

We successfully used this methodology to create a UX design process for HRI that both assists in creating a viable application and an enhanced user experience in several domains, such as banks, shopping centres and airports [7, 12, 9]. In addition, our suggested UX design process have been used to easily design experiments to test psychological and sociological effects of HRI for the considered applications.

In this perspective, RoboCup@Home is a significant opportunity to showcase our robotics commercial applications, to gather feedback from robotics community and to test the HRI of the designed systems.

A new agile next-generation cognitive architecture, BUNJI, is being developed for RoboCup 2019 based on the insights gathered during our experience at RoboCup 2017 & 2018, see Fig. 2. BUNJI means friend in several Australian Aboriginal languages.

In order to allow a range of high-level reasoning approaches a common modular low level layer is needed. We call this layer, Skills (saying a sentence, for example). Composing one or more skills creates a Capability (e.g., answering a question).

These Skills and Capabilities can be orchestrated to create plans to achieve tasks, from subtasks such as a RoboCup test (which itself can become a Capability) to a full competition test.

Capabilities are internally represented as SMACH State Machines[25] so we can take advantage of their properties and pre-existing tools.

In order to move away from the simple state machine based approach, which plagues the league we make use of a whiteboard of common knowledge called Beliefs which our reasoning engine uses in conjunction with the Capabilities of the robot to dynamically create plans to solve the RoboCup tests as if they were General Purpose Service Robot tasks.

These plans can be represented in what we call Plan file format, which can be used to be read by humans and also to be debugged with our visual interfaces. Plans can also be considered Behaviours.

Initially, easy tests may be graphically solved by using our block-like interface, once tests require a more cognitive approach other engines can be used.

We use the following external sources of software:

The Robot Operating System ROS a collection of open source tools, libraries, and conventions for developing robot behaviour across a wide variety of robotic platforms.
Machine learning algorithms from Dlib a collection of open source machine learning algorithms that can be in a wide range of domains including robotics, embedded devices, mobile phones, and large high performance computing environments.
Keras Python API for enabling Deep Learning models on Pepper Robots.

Although our architecture supports external and cloud services we do not intend to use them at competition many due to the network traffic which typically interferes. Instead we prefer to run all behaviours on-board the robot.

4 Reusability for Other Research Groups

Our research lab has placed public dissemination and re-usability as a priority in its research efforts. New projects are, by default, made publicly available.

Our research lab Github profile is growing rapidly hosting 39 repositories, in addition to the 230 repositories associated with the individual Github profiles of lab's members.

We currently plan to continue our open position on code release as part of the RoboCup@Home campaign and will develop and release stable and documented code around the following themes:

1. A rapid experimentation and development framework based on Python, ROS and a range of web toolkits.
1. Cognitive architectures for intelligent social and socially-motivated behaviour.
1. Behavioural and gestural control system that combine web interfaces with intelligent behaviours.
1. Visual systems optimised for fluent social behaviour.
1. Visual tools for easy debugging of complex behaviours.

As the robot platform Pepper is standardised so the re-usability is assured. Also the ROS software components can be shared with other robots.

The software development methodology used in our research lab is also a topic of active research and is to be shared with the league too soon.

5 Applicability of the Approach in the Real World

Our research is conducted with a direct and immediate focus on real-world applications. We conducted research with Australia's largest bank and with other major commercial partners. Currently, we are investigating the deployment of social robots applications in public spaces by collaborating with Australia's government bodies and public hospitals. This research agenda is intended to have commercial applications and affords us venues to test research in real world applications, thus translating research into societal benefits. Our strategic web of partnerships enables us to do this without impacting our ability to publish original research and make our source code available to other research groups around the world.

Also RoboCup@Home focuses in real life home environment situations so every skill developed for the contest has a direct application in the real world.

6 Conclusions and Future Work

Our team has more than 15 years experience in robotics research and development, with a particular focus on social robots co-existing in human-centric spaces and co-operating with people. We have participated at RoboCup since 2003 and the recognise that @Home League offers unparalleled opportunity to contribute and to test our social robot applications.

In addition, Sammy Pfeiffer is an active member of the Technical Committee for second year in a row. He is looking forward to working with committee members and team leaders to improve the league and make it progressively more sociable.

Our team's aim is to include focus on the social and HRI aspects in the upcoming competition for this league, so that RoboCup@Home can positively enrich specific areas of robotics challenges that it was designed to meet.

Future work will include research oriented to further develop and extend the capabilities of our novel skill-based oriented system, BUNJI, to allow even more agile development embracing continuous integration to ensure robustness. Also empowering more human-like approaches enabling robots to take decisions and reactively change plans to solve unexpected situations.

References

Mary-Anne Williams, William Judge, Jonathan Vitale, Meg Tonkin, Benjamin Johnston, and Amit Kumar Pandey. 1st workshop on human-robot engagement in the home, workplace and public space. In International Joint Conference on Artificial Intelligence, 2017.
Hugo Romat, Mary-Anne Williams, Xun Wang, Benjamin Johnston, and Henry Bard. Natural human-robot interaction using social cues. In The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 503–504. IEEE Press, 2016.
John-John Cabibihan, Mary-Anne Williams, and Reid G Simmons. When robots engage humans. IJ Social Robotics, 6(3):311–313, 2014.
Harry Surden and Mary-Anne Williams. Technological opacity, predictability, and self-driving cars. Cardozo Law Review, 38(1):121–182, 2016.
Nima Ramezani and Mary-Anne Williams. Smooth robot motion with an optimal redundancy resolution for pr2 robot based on an analytic inverse kinematic solution. In Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, pages 338–345. IEEE, 2015.
Shaukat Abidi, Massimo Piccardi, and Mary-Anne Williams. Static action recognition by efficient greedy inference. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1–8. IEEE, 2016.
Jonathan Vitale, Meg Tonkin, Xun Wang, Suman Ojha, and Mary-Anne Williams. Privacy by design in machine learning data collection: Ux experimentation in hri setting. In AAAI Spring Symposia 2017, Designing the User Experience of Machine Learning Systems, submitted.
Jonathan Vitale, Meg Tonkin, Sarita Herse, Suman Ojha, Jesse Clark, Mary-Anne Williams, Xun Wang, and William Judge. Be more transparent and users will like you: A robot privacy and user experience design experiment. In Proceedings of 2018 ACM/IEEE International Conference on Human Robot Interaction, Chicago, IL, USA, March 5–8, 2018, 2018.
Meg Tonkin, Jonathan Vitale, Suman Ojha, Jesse Clark, Sammy Pfeiffer, William Judge, Xun Wang, and Mary-Anne Williams. Embodiment, privacy and social robots: May i remember you? In The 9th International Conference on Social Robotics, November 2017.
Sarita Herse, Jonathan Vitale, Daniel Ebrahimian, Meg Tonkin, Suman Ojha, Sidra Sidra, Benjamin Johnston, Sophie Phillips, Siva Leela Krishna Chand Gudi, Jesse Clark, William Judge, and Mary-Anne Williams. Bon appetit! robot persuasion for food recommendation. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pages 125–126. ACM, 2018.
Sarita Herse, Jonathan Vitale, Meg Tonkin, Daniel Ebrahimian, Suman Ojha, Benjamin Johnston, William Judge, and Mary-Anne Williams. Do you trust me, blindly? factors influencing trust towards a robot recommender system. In The 27th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2018. IEEE, 2018.
Meg Tonkin, Jonathan Vitale, Suman Ojha, Mary-Anne Williams, Paul Fuller, and William Judge. Would you like to sample? Robot engagement in a shopping centre. In The 26th IEEE International Symposium on Robot and Human Interactive Communication, August 2017.
Meg Tonkin, Jonathan Vitale, Sarita Herse, Mary-Anne Williams, William Judge, and Xun Wang. Design methodology for the ux of hri: A field study of a commercial social robot at an airport. In Proceedings of 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, March 5-8, 2018 (HRI '18). ACM, March 2018.
Mary-Anne Williams. Robot social intelligence. In International Conference on Social Robotics, pages 45–55. Springer, 2012.
Jonathan Vitale, Mary-Anne Williams, and Benjamin Johnston. Socially Impaired Robots: Human Social Disorders and Robots' Socio-Emotional Intelligence, chapter Social Robotics: 6th International Conference, ICSR 2014, Sydney, NSW, Australia, October 27-29, 2014. Proceedings, pages 350–359. Springer International Publishing, 2014.
Jonathan Vitale, Mary-Anne Williams, Benjamin Johnston, and Giuseppe Boccignone. Affective facial expression processing via simulation: A probabilistic model. Biologically Inspired Cognitive Architectures, 10:30 – 41, 2014.
Suman Ojha, Jonathan Vitale, and Williams Mary-Anne. A domain-independent approach of cognitive appraisal augmented by higher cognitive layer of ethical reasoning. In Annual Meeting of the Cognitive Science Society, 2017.
Rony Novianto and Mary-Anne Williams. Emotion in robot decision making. Advances in Intelligent Systems and Computing, 10(Vol. 447):221 – 232, 2017.
Mary-Anne Williams and Nick Dawson. United nations impact of ai and robotics on human rights. Sustainable Development and Peace and Security Report, 2018.
Siddharth Agrawal and Mary-Anne Williams. Robot authority and human obedience: A study of human behaviour using a robot security guard. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, HRI '17, pages 57–58, New York, NY, USA, 2017. ACM.
Siddharth Agrawal and Mary-Anne Williams. Would you obey an aggressive robot: A human-robot interaction field study. In The 27th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2018. IEEE, 2018.
Adam Wiggins. The twelve-factor app. https://12factor.net/, 2007.
Jeff Gothelf. Lean UX: Applying lean principles to improve user experience. " O'Reilly Media, Inc.", 2013.
Eric B Hekler, Predrag Klasnja, William T Riley, Matthew P Buman, Jennifer Huberty, Daniel E Rivera, and Cesar A Martin. Agile science: creating useful products for behavior change in the real world. Translational behavioral medicine, 6(2):317–328, 2016.
Jonathan Bohren and Steve Cousins. The smach high-level executive [ros news]. IEEE Robotics & Automation Magazine, 17(4):18–20, 2010.

UTS Unleashed! 2019 RoboCup@Home Social SPL