UT Austin Villa RoboCup@Home Domestic Standard Platform League Team Description Paper

Justin W. Hart, Peter Stone, Andrea Thomaz, Scott Niekum

Department of Computer Science, University of Texas at Austin; Department of Electrical and Computer Engineering, University of Texas at Austin

http://www.cs.utexas.edu/AustinVilla/athome/Team2017/

Abstract Our general approach to the RoboCup@Home competition is one of integrating the HSR into the existing robotics research infrastructure of three collaborating laboratories here at the University of Texas at Austin. These laboratories focus on research toward the development of artificial intelligence, machine learning, and human-robot interaction technologies which provide services to and interact with human users in both a simulated home environment and in a live deployment known as the Building-Wide Intelligence (BWI) Project in the Gates-Dell Computer Science Complex at the University of Texas at Austin. By incorporating the HSR into our existing ecosystem, we will be able to rapidly deploy relevant components from our ongoing research into its software infrastructure for the competition. Incorporation into our existing research also guarantees the reusability of software developed for the competition as it is integrated into the large software deployment of the BWI Project. Significant portions of the code developed by our group for competition will be released as components of the BWI infrastructure code as open source software through ROS.org. Relevant current work by our groups involves research in natural language processing, multimodal language learning, learning from demonstration, human activity recognition, planning, and probabilistic and symbolic reasoning. In the past year, significant work has been performed on domestic tasks such as storing groceries, and language tasks such as natural language processing and the performance of natural language tasks from speech.

1 Introduction

The UT Austin Villa team, from the University of Texas at Austin, has a long and successful history of RoboCup participation. In 2016, we finished 2nd place in the soccer standard platform league (SPL), and won first place in the 3D simulation league for the 5th time in 6 years. Although we have also pursued research on domestic and service robots for many years, we have only once participated in RoboCup@Home, finishing in 2nd place in 2007. This application brings together the laboratories of UT Austin Villa founder and team leader of Prof. Peter Stone with those of two colleagues at UT Austin who are experts in interactive robotics, but who have never participated in RoboCup: Prof. Andrea Thomaz and Prof. Scott Niekum. Our entry in the 2017 RoboCup@Home competition will unite these three groups to focus our complementary research in the areas of machine learning, service robotics, and human robot interaction towards an entry in the RoboCup@Home SPL using the Toyota Human Support Robot (HSR).

2 Approach

The laboratories of Stone, Thomaz, and Niekum currently have active research programs that are directly relevant to the challenge put forth by the RoboCup@Home competition. Prof. Stone's laboratory is pursuing the Building-Wide Intelligence (BWI) project, which includes a group of several human-interactive robots, and which has the long-term objective of enabling a team of robots to achieve longterm autonomy within the rich socially interactive environment of the UT Austin Computer Science Department in the Gates-Dell Complex. Profs. Thomaz and Niekum have performed much of their past research in home-like environments and have an existing shared lab space set up as an apartment with a living room, dining room, and kitchen. Currently, our groups use similar, but distinct robot platforms. The BWI project uses a fleet of SegBots, Figure 1a, custom designed mobile robots built atop Segway RMP bases. In the past year BWI has been performing robot learning experiments with a SegBot with Kinova Mico arm mounted, Figure 1b. Thomaz and Niekum use a pair of similar custom robots atop an omni-drive base, both with arms, Figure 1c. The basic approach that our team will take in the RoboCup@Home competition is to integrate the HSR platform into the fleet of robots used in the Building-Wide Intelligence project, and to use development on the shared common platform of the Toyota HSR towards the RoboCup@Home competition as a means of integrating research across our laboratories.

Among the central goals of our research is the development of robotic systems capable of long-term autonomous human-robot interaction, automated planning, reasoning, and learning in complex real-world environments. By defining a novel competition framework situated within a domestic environment that uniquely exposes the challenges of this proposition, RoboCup@Home is an ideal setting for advancing the state of the art in long-term autonomy for human-interactive robots, and presents a both a task framework that is harmonized with our goals and a standard set of tasks against which we can compare our platform to other systems. With this motivation in mind, it is our goal to combine the expertise of Profs. Stone, Thomaz, and Niekum in the areas of reinforcement learning, human-robot interaction, and learning from demonstration towards a full solution to the RoboCup@Home challenge, along with their laboratories. Our team brings together postdocs, graduate students, and undergraduate students from each of our groups to enable the robot to interact seemlessly with people in the environment, and to learn behaviors ranging from object manipulation to robust environmental perception, to high-level task sequencing.

By leveraging our existing software and incorporating our HSR into the BWI infrastructure, we hope to gain a jump start on the competition. We will also be able to make use of the infrastructure which we have in place for experiments on human-robot interaction in domestic and public spaces. Thomaz and Niekum's shared laboratory space is located in the same building as the BWI project, allowing us to test our system in both an open, interactive setting with many people to potentially interact with, and more controlled home-like settings. We expect that participation in the RoboCup@Home will boost our research efforts through collaboration between our groups, the sharing of ideas in an environment of friendly competition, and the advancements required to accomplish the tasks of the competition itself. We expect to make advancements on topics ranging from activity recognition, to robust perception, to learned planning and navigation, to general human-robot interaction.

3 Research Focus

Our main research foci from a technical perspective are on reinforcement learning, especially as applied to real robots, multi-robot systems, human-robot interaction, and learning from demonstration. However we also have a strong history of creating integrated, fully autonomous systems with the idea that integration is an important and valuable research focus in and of itself. We view RoboCup@Home as a perfect setting for both stretching our core research competencies and for supporting our commitment to integration.

Examples of our past research innovations related to RoboCup@Home are provided in Section 4.

4 Innovative Technology

The collective breadth of our research has enabled contributions to a variety of AI sub-areas beyond HRI, including AI planning, knowledge representation and reasoning, natural language processing, and machine learning. Here we provide a brief description of some of our published research, some of which will form the core of our RoboCup@Home approach, and some of which we expect to be substantially extended and improved through our participation in RoboCup@Home.

Understanding natural language requests

One of the most natural forms of human-robot interaction for humans is through natural language. However natural language processing remains a challenging research area within AI, and intelligent service robots should be able to efficiently and accurately understand commands from human users speaking in natural language.

We introduced research contributions pertaining to language learning to facilitate on-line improvement of the robots' understanding of spoken commands. We use a dialog agent embodied in a BWIBot to communicate with users through natural language and improve language understanding over time using data from these conversations [1]. By learning from conversations, our approach can recognize more complex language than keywordbased approaches without needing the large-scale, hand-annotated training data associated with complex language understanding tasks [1].

To this end, we trained a semantic parser with a tiny set of expressions paired with robot goals. The natural language understanding component of our system is this semantic parser together with a conversational dialog agent. The dialog agent keeps track of the system's partial understanding of the goal the user is trying to convey and asks clarification questions to refine that understanding.

Grounded multimodal language learning

Often it is necessary for a robot to ground language using its own perception and actions with respect to objects. Consider the case where a human asks a service robot, "Please bring me the full red bottle". To fulfill such a request, a robot would need to detect objects in its environment and determine whether the words "full", "red", and "bottle" match a particular object detection. Furthermore, such a task cannot be solved using static visual object recognition methods as detecting whether an object is full or empty may often require the robot to perform a certain action on it (e.g., lift the object to measure the force it exerts on the arm).

This research contribution [2] focused on solving the symbol grounding problem [3], a longstanding challenge in AI, where language is grounded using the robot's perception and action. To address this problem, we enable a robot to undergo two distinct developmental stages:

• 1. Object Exploration Stage the robot interacts with objects using a set of exploratory behaviors designed to produce different kinds of multi-modal feedback.
• 2. Social Learning Stage the robot interacts with humans in order to learn mappings from its sensorimotor experience with objects to words that can be used to described the objects.

Learning multi-step tasks from demonstration

Much previous research in learning from demonstration has focused on learning policies—mappings from states to actions—for skills such as a tennis swing or humanoid walking. However, these techniques typically perform poorly for complex, multi-step tasks that cannot be represented monolithically with a single policy. We developed a series of LfD algorithms that, for the first time, allowed a robot to learn the structure of complex tasks such as IKEA furniture assembly from a small number of demonstrations [4–8]. This research led to state-of-the-art Bayesian nonparametric and control techniques that were able to automatically identify an appropriate number of skills from task demonstrations [7], infer the goals of each skill [9, 6], construct controllers to accomplish these goals, and intelligently sequence these controllers based on perceptual feedback [4, 5, 8].

Robot-centric human activity recognition

For a robot to effectively function in a human-inhabited environment, it would be useful for it to be aware of the activities and intentions of humans around it based on its own observations. For example, consider the case where a robot is navigating a crowded environment such as an undergraduate computer lab. If the robot could recognize when a person needs help, or when a person is trying to approach or engage it (or avoid it), its social and navigational skills would improve dramatically. For this purpose, we contributed a novel approach which explores how human activity can be recognized, making it possible for a robot to understand the intent of humans in its vicinity.

Our robot uses its autonomous navigation capability in a large, unstructured, and human-inhabited environment. The activities learned by our robot were performed spontaneously by many different people who interacted with (or were observed by) the robot, as opposed to the standard methodology of asking study participants to perform certain actions. In contrast to classic computer vision approaches, our system uses both visual and non-visual cues when recognizing the activities of humans that it interacts with [10].

Planning using action language BC

General purpose planning domain descriptions can be written using various modes. Action languages such as BC are attractive in task planning for mobile robots because they solve the frame problem, which states that many axioms are necessary to express that things in the environment do not change arbitrarily [11]. For example, when a robot picks up an object from the table, it does not change the location of a different object on the table. BC solves this problem by easily expressing rules of inertia. In addition, BC can solve the ramification problem, which is concerned with the indirect consequences of an action [12]. For example, when a robot picks up a tray from the table, it indirectly changes the location of any object on the tray. BC can also easily express indirect and recursive effects of actions.

We have demonstrated how action language BC can be used for robot task planning in domains requiring planning in the presence of missing information and indirect/recursive action effects [13]. While we demonstrate using BC to express a mail collection task, the overall methodology is applicable to any other planning domains that require: recursive and indirect action effects, defeasible reasoning, and acquiring previously unknown knowledge through human-robot interaction. In addition, we also demonstrated how answer set planning under action costs [14] can be applied to robot task planning in conjunction with BC [13].

Integrating probabilistic and symbolic reasoning

While action language BC can express defeasible reasoning, it cannot express probabilities, and consequently cannot be used for stochastic planning. We therefore introduced a method for robots to efficiently and robustly fulfill service requests in human-inhabited environments by simultaneously reasoning about commonsense knowledge expressed using defeasible reasoning and computing plans under uncertainty [15]. We illustrated this planning paradigm using a Spoken Dialog System (sds), where the robot identifies a spoken shopping request from the user in the presence of noise and/or incomplete instructions. The goal of the system is to identify the shopping request as quickly as possible while minimizing the cost of asking questions. Once confirmed, the robot attempts to deliver the item. While this planning paradigm is described in the context of an sds, it can just as easily be applied to other stochastic planning problems as well.

A much more extensive list of relevant research contributions, as well as links to our papers, is available from the application webpage.

5 Re-Usability

Throughout our development, we will continue our strong tradition of releasing well-documented, self-contained behavior modules for components of autonomous robots that can be used by other research groups to enhance their own research. As listed with links on the propoosal webpage, these include:

– Our BWI code repository provides an open source suite of ROS packages, fully integrated into an architecture for service robots that operate in dynamic and unstructured human-inhabited environments. It has been built on top of the Robot Operating System (ROS) middleware framework [16], and is available open-source at https://github.com/utexas-bwi/. This software architecture provides a hierarchical layered approach for controlling autonomous robots, where layers in the hierarchy provide different granularities of control. Specifically, this architecture includes navigation software that allows a mobile robot to move autonomous inside a building, while being able to switch 2D navigational maps when using the elevator to move to a different floor. A symbolic navigation module is built on top of this autonomous navigation module which allows the robot to navigate to prespecified doors, rooms, and objects in the environment. From a high-level perspective, the software architecture includes planning and reasoning modules that allow the robot to execute high level tasks, such as delivering an object from one part of the building to another, using a complex sequence of symbolic actions.

• Our TEXPLORE code provides an open-source package for reinforcement learning on real robots.
• Our ar track alvar ROS package has become a community standard for tagbased perception
• Our ROS implementation of Dynamic Movement Primitives has become a popular tool for learning from demonstration.
• We have made research code available for:
  • Bayesian changepoint detection,
  • active articulated model estimation, and
  • Bayesian nonparametric skill learning from demonstration.

Our team has also released several packages based on our robot soccer software that have been used by other teams. These include source code for our championship teams in both the 3D simulation league and the soccer standard platform league, as detailed (with links) on our application webpage.

We are fully committed to sharing any useful innovations that result from our participation in RoboCup@Home by releasing self-contained, fully documented modules for component tasks that will be useful for future RoboCup@Home participants, and more broadly by AI and robotics researchers around the world.

6 Applicability to the Real World

We will test our research both in the controlled setting of our lab with a living room, dining room, and kitchen and in the open, dynamic, uncontrolled setting of the Building-Wide Intelligence project that enables any visitor to the CS building to interact with the robots. The former environment will enable development within a setting that closely resembles the RoboCup@Home test arena, while the latter will ensure that our software is robust and applicable to the real world.

All of our innovations related to the RoboCup@Home competition will be fully evaluated and tested using established HRI experimentation methods, and incorporated into our ongoing research that will be submitted for publication in leading AI and robotics conferences and journals. We are committed to our research related to RoboCup@Home contributing to our overall ongoing research agenda on creating fully autonomous, human-interactive robots that learn and interact in the real world.

References

1. Thomason, J., Zhang, S., Mooney, R., Stone, P.: Learning to interpret natural language commands through human-robot dialog. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI). (2015) 1923–1929
2. Thomason, J., Sinapov, J., Svetlik, M., Stone, P., Mooney, R.: Learning multimodal grounded linguistic semantics by playing "I spy". In: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI). (2016) 3477–3483
3. Harnad, S.: The symbol grounding problem. Physica D: Nonlinear Phenomena 42 (1990) 335–346
4. Niekum, S.: Semantically grounded learning from unstructured demonstrations. PhD thesis, Department of Computer Science, University of Massachusetts Amherst (2013)
5. Niekum, S., Chitta, S., Barto, A.G., Marthi, B., Osentoski, S.: Incremental semantically grounded learning from demonstration. In: Robotics: Science and Systems (RSS). Volume 9. (2013)
6. Niekum, S., Osentoski, S., Atkeson, C.G., Barto, A.G.: Online bayesian changepoint detection for articulated motion models. In: IEEE International Conference on Robotics and Automation (ICRA). (2015)
7. Niekum, S., Osentoski, S., Konidaris, G., Barto, A.G.: Learning and generalization of complex tasks from unstructured demonstrations. In: Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, IEEE (2012)
8. Niekum, S., Osentoski, S., Konidaris, G., Chitta, S., Marthi, B., Barto, A.G.: Learning grounded finite-state representations from unstructured demonstrations. The International Journal of Robotics Research 34 (2015) 131–157
9. Niekum, S., Barto, A.G.: Clustering via dirichlet process mixture models for portable skill discovery. In: Advances in Neural Information Processing Systems 24 (NIPS). (2011)
10. Gori, I., Sinapov, J., Khante, P., Stone, P., Aggarwal, J.: Robot-centric activity recognition 'in the wild'. In: Social Robotics. Springer International Publishing (2015) 224–234
11. McCarthy, J., Hayes, P.: Some philosophical problems from the standpoint of artificial intelligence. In: Machine Intelligence. Edinburgh University Press (1969)
12. Finger, J.: Exploiting Constraints in Design Synthesis. PhD thesis, Stanford University (1986)
13. Khandelwal, P., Yang, F., Leonetti, M., Lifschitz, V., Stone, P.: Planning in action language bc while learning action costs for mobile robots. In: International Conference on Automated Planning and Scheduling (ICAPS). (2014)
14. Eiter, T., Faber, W., Leone, N., Pfeifer, G., Polleres, A.: Answer set planning under action costs. Journal of Artificial Intelligence Research (JAIR) (2003)
15. Zhang, S., Stone, P.: Corpp: Commonsense reasoning and probabilistic planning, as applied to dialog with a mobile robot. In: AAAI. (2015) 1394–1400
16. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA workshop on open source software. Volume 3. (2009) 5