Author name cluster

Sonia Chernova

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

48 papers

2 author rows

IROS Conference 2023 Conference Paper

ConSOR: A Context-Aware Semantic Object Rearrangement Framework for Partially Arranged Scenes

Kartik Ramachandruni
Max Zuo
Sonia Chernova

Object rearrangement is the problem of enabling a robot to identify the correct object placement in a complex environment. Prior work on object rearrangement has explored a diverse set of techniques for following user instructions to achieve some desired goal state. Logical predicates, images of the goal scene, and natural language descriptions have all been used to instruct a robot in how to arrange objects. In this work, we argue that burdening the user with specifying goal scenes is not necessary in partially-arranged environments, such as common household settings. Instead, we show that contextual cues from partially arranged scenes (i. e. , the placement of some number of pre-arranged objects in the environment) provide sufficient context to enable robots to perform object rearrangement without any explicit user goal specification. We introduce ConSOR, a Context-aware Semantic Object Rearrangement framework that utilizes contextual cues from a partially arranged initial state of the environment to complete the arrangement of new objects, without explicit goal specification from the user. We demonstrate that ConSOR strongly outperforms two baselines in generalizing to novel object arrangements and unseen object categories. The code and data are available at https://github.com/kartikvrama/consor.

IJCAI Conference 2023 Conference Paper

Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework (Extended Abstract)

Evana Gizzi
Lakshmi Nair
Sonia Chernova
Jivko Sinapov

Creative Problem Solving (CPS) is a sub-area within artificial intelligence that focuses on methods for solving off-nominal, or anomalous problems in autonomous systems. Despite many advancements in planning and learning in AI, resolving novel problems or adapting existing knowledge to a new context, especially in cases where the environment may change in unpredictable ways, remains a challenge. To stimulate further research in CPS, we contribute a definition and a framework of CPS, which we use to categorize existing AI methods in this field. We conclude our survey with open research questions, and suggested future directions.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding

Devleena Das
Sonia Chernova
Been Kim

As more non-AI experts use complex AI systems for daily tasks, there has been an increasing effort to develop methods that produce explanations of AI decision making that are understandable by non-AI experts. Towards this effort, leveraging higher-level concepts and producing concept-based explanations have become a popular method. Most concept-based explanations have been developed for classification techniques, and we posit that the few existing methods for sequential decision making are limited in scope. In this work, we first contribute a desiderata for defining ``concepts'' in sequential decision making settings. Additionally, inspired by the Protege Effect which states explaining knowledge often reinforces one's self-learning, we explore how concept-based explanations of an RL agent's decision making can in turn improve the agent's learning rate, as well as improve end-user understanding of the agent's decision making. To this end, we contribute a unified framework, State2Explanation (S2E), that involves learning a joint embedding model between state-action pairs and concept-based explanations, and leveraging such learned model to both (1) inform reward shaping during an agent's training, and (2) provide explanations to end-users at deployment for improved task performance. Our experimental validations, in Connect 4 and Lunar Lander, demonstrate the success of S2E in providing a dual-benefit, successfully informing reward shaping and improving agent learning rate, as well as significantly improving end user task performance at deployment time.

JAIR Journal 2022 Journal Article

Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework

Evana Gizzi
Lakshmi Nair
Sonia Chernova
Jivko Sinapov

Creative Problem Solving (CPS) is a sub-area within Artificial Intelligence (AI) that focuses on methods for solving off-nominal, or anomalous problems in autonomous systems. Despite many advancements in planning and learning, resolving novel problems or adapting existing knowledge to a new context, especially in cases where the environment may change in unpredictable ways post deployment, remains a limiting factor in the safe and useful integration of intelligent systems. The emergence of increasingly autonomous systems dictates the necessity for AI agents to deal with environmental uncertainty through creativity. To stimulate further research in CPS, we present a definition and a framework of CPS, which we adopt to categorize existing AI methods in this field. Our framework consists of four main components of a CPS problem, namely, 1) problem formulation, 2) knowledge representation, 3) method of knowledge manipulation, and 4) method of evaluation. We conclude our survey with open research questions, and suggested directions for the future.

PDF Details DOI

IROS Conference 2022 Conference Paper

Explainable Knowledge Graph Embedding: Inference Reconciliation for Knowledge Inferences Supporting Robot Actions

Angel Andres Daruna
Devleena Das
Sonia Chernova

Learned knowledge graph representations supporting robots contain a wealth of domain knowledge that drives robot behavior. However, there does not exist an inference reconciliation framework that expresses how a knowledge graph representation affects a robot's sequential decision making. We use a pedagogical approach to explain the inferences of a learned, black-box knowledge graph representation, a knowledge graph embedding. Our interpretable model uses a decision tree classifier to locally approximate the predictions of the black-box model and provides natural language explanations interpretable by non-experts. Results from our algorithmic evaluation affirm our model design choices, and the results of our user studies with non-experts support the need for the proposed inference reconciliation framework. Critically, results from our simulated robot evaluation indicate that our explanations enable non-experts to correct erratic robot behaviors due to nonsensical beliefs within the black-box.

IROS Conference 2021 Conference Paper

An Interleaved Approach to Trait-Based Task Allocation and Scheduling

Glen Neville
Andrew Messing
Harish Ravichandar
Seth Hutchinson 0001
Sonia Chernova

To realize effective heterogeneous multi-robot teams, researchers must leverage individual robots’ relative strengths and coordinate their individual behaviors. Specifically, heterogeneous multi-robot systems must answer three important questions: who (task allocation), when (scheduling), and how (motion planning). While specific variants of each of these problems are known to be NP-Hard, their interdependence only exacerbates the challenges involved in solving them together. In this paper, we present a novel framework that interleaves task allocation, scheduling, and motion planning. We introduce a search-based approach for trait-based time-extended task allocation named Incremental Task Allocation Graph Search (ITAGS). In contrast to approaches that solve the three problems in sequence, ITAGS’s interleaved approach enables efficient search for allocations while simultaneously satisfying scheduling constraints and accounting for the time taken to execute motion plans. To enable effective interleaving, we develop a convex combination of two search heuristics that optimizes the satisfaction of task requirements as well as the makespan of the associated schedule. We demonstrate the efficacy of ITAGS using detailed ablation studies and comparisons against two state-of-the-art algorithms in a simulated emergency response domain.

IROS Conference 2021 Conference Paper

Desperate Times Call for Desperate Measures: Towards Risk-Adaptive Task Allocation

Max Rudolph
Sonia Chernova
Harish Ravichandar

Multi-robot task allocation (MRTA) problems involve optimizing the allocation of robots to tasks. MRTA problems are known to be challenging when tasks require multiple robots and the team is composed of heterogeneous robots. These challenges are further exacerbated when we need to account for uncertainties encountered in the real-world. In this work, we address coalition formation in heterogeneous multi-robot teams with uncertain capabilities. We specifically focus on tasks that require coalitions to collectively satisfy certain minimum requirements. Existing approaches to uncertainty-aware task allocation either maximize expected pay-off (risk-neutral approaches) or improve worst-case or near-worst-case outcomes (risk-averse approaches). Within the context of our problem, we demonstrate the inherent limitations of unilaterally ignoring or avoiding risk and show that these approaches can in fact reduce the probability of satisfying task requirements. Inspired by models that explain foraging behaviors in animals, we develop a risk-adaptive approach to task allocation. Our approach adaptively switches between risk-averse and risk-seeking behavior in order to maximize the probability of satisfying task requirements. Comprehensive numerical experiments conclusively demonstrate that our risk-adaptive approach outperforms risk-neutral and risk-averse approaches. We also demonstrate the effectiveness of our approach using a simulated multi-robot emergency response scenario.

IROS Conference 2021 Conference Paper

Learning Navigation Skills for Legged Robots with Learned Robot Embeddings

Joanne Truong
Denis Yarats
Tianyu Li 0005
Franziska Meier
Sonia Chernova
Dhruv Batra
Akshara Rai

Recent work has shown results on learning navigation policies for idealized cylinder agents in simulation and transferring them to real wheeled robots. Deploying such navigation policies on legged robots can be challenging due to their complex dynamics, and the large dynamical difference between cylinder agents and legged systems. In this work, we learn hierarchical navigation policies that account for the low-level dynamics of legged robots, such as maximum speed, slipping, contacts, and learn to successfully navigate cluttered indoor environments. To enable transfer of policies learned in simulation to new legged robots and hardware, we learn dynamics-aware navigation policies across multiple robots with robot-specific embeddings. The learned embedding is optimized on new robots, while the rest of the policy is kept fixed, allowing for quick adaptation. We train our policies across three legged robots in simulation - 2 quadrupeds (A1, AlienGo) and a hexapod (Daisy). At test time, we study the performance of our learned policy on two new legged robots in simulation (Laikago, 4-legged Daisy), and one real-world quadrupedal robot (A1). Our experiments show that our learned policy can sample-efficiently generalize to previously unseen robots, and enable sim-to-real transfer of navigation policies for legged robots.

IROS Conference 2021 Conference Paper

Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failures

Devleena Das
Sonia Chernova

When interacting in unstructured human environments, occasional robot failures are inevitable. When such failures occur, everyday people, rather than trained technicians, will be the first to respond. Existing natural language explanations hand-annotate contextual information from an environment to help everyday people understand robot failures. However, this methodology lacks generalizability and scalability. In our work, we introduce a more generalizable semantic explanation framework. Our framework autonomously captures the semantic information in a scene to produce semantically descriptive explanations for everyday users. To generate failure-focused explanations that are semantically grounded, we lever-ages both semantic scene graphs to extract spatial relations and object attributes from an environment, as well as pairwise ranking. Our results show that these semantically descriptive explanations significantly improve everyday users’ ability to both identify failures and provide assistance for recovery than the existing state-of-the-art context-based explanations.

AAMAS Conference 2021 Conference Paper

STRATA: Unified Framework for Task Assignments in Large Teams of Heterogeneous Agents

Harish Ravichandar
Kenneth Shaw
Sonia Chernova

Large teams of heterogeneous agents have the potential to solve complex multi-task problems that are intractable for a single agent working independently. However, solving complex multi-task problems requires leveraging the relative strengths of the different kinds of agents in the team. We present Stochastic TRAit-based Task Assignment (STRATA), a unified framework that models large teams of heterogeneous agents and performs effective task assignments. Specifically, given information on which traits (capabilities) are required for various tasks, STRATA computes the assignments of agents to tasks such that the trait requirements are achieved. Inspired by prior work in robot swarms and biodiversity, we categorize agents into different species (groups) based on their traits. We model each trait as a continuous variable and differentiate between traits that can and cannot be aggregated from different agents. STRATA is capable of reasoning about both species-level and agentlevel variability in traits. We illustrate the necessity and effectiveness of STRATA using detailed numerical simulations and in a capture-the-flag game environment.

IROS Conference 2021 Conference Paper

Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees

Muhammad Asif Rana
Anqi Li 0001
Dieter Fox
Sonia Chernova
Byron Boots
Nathan D. Ratliff

Generating robot motion that fulfills multiple tasks simultaneously is challenging due to the geometric constraints imposed on the robot. In this paper, we propose to solve multi-task problems through learning structured policies from human demonstrations. Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces. The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned. We derive an end-to-end learning objective that is suitable for the multi-task problem, emphasizing the distance between generated motions and demonstrations measured on task spaces. Furthermore, the motion generated from the learned policy class is guaranteed to be stable. We validate the effectiveness of our proposed learning framework through qualitative and quantitative evaluations on three robotic tasks on a 7-DOF Rethink Sawyer robot.

ICRA Conference 2021 Conference Paper

Towards Robust One-shot Task Execution using Knowledge Graph Embeddings

Angel Andres Daruna
Lakshmi Nair 0001
Weiyu Liu
Sonia Chernova

Requiring multiple demonstrations of a task plan presents a burden to end-users of robots. However, robustly executing tasks plans from a single end-user demonstration is an ongoing challenge in robotics. We address the problem of one-shot task execution, in which a robot must generalize a single demonstration or prototypical example of a task plan to a new execution environment. Our approach integrates task plans with domain knowledge to infer task plan constituents for new execution environments. Our experimental evaluations show that our knowledge representation makes more relevant generalizations that result in significantly higher success rates over tested baselines. We validated the approach on a physical platform, which resulted in the successful generalization of initial task plans to 38 of 50 execution environments with errors resulting from autonomous robot operation included.

IROS Conference 2020 Conference Paper

Anticipatory Human-Robot Collaboration via Multi-Objective Trajectory Optimization

Abhinav Jain 0002
Daphne Chen
Dhruva Bansal
Sam Scheele
Mayank Kishore
Hritik Sapra
David Kent 0001
Harish Ravichandar

We address the problem of adapting robot trajectories to improve safety, comfort, and efficiency in humanrobot collaborative tasks. To this end, we propose CoMOTO, a trajectory optimization framework that utilizes stochastic motion prediction to anticipate the human's motion and adapt the robot's joint trajectory accordingly. We design a multiobjective cost function that simultaneously optimizes for i) separation distance, ii) visibility of the end-effector, iii) legibility, iv) efficiency, and v) smoothness. We evaluate CoMOTO against three existing methods for robot trajectory generation when in close proximity to humans. Our experimental results indicate that our approach consistently outperforms existing methods over a combined set of safety, comfort, and efficiency metrics.

IROS Conference 2020 Conference Paper

Approximated Dynamic Trait Models for Heterogeneous Multi-Robot Teams

Glen Neville
Harish Ravichandar
Kenneth Shaw
Sonia Chernova

To realize effective heterogeneous multi-agent teams, we must be able to leverage individual agents' relative strengths. Recent work has addressed this challenge by introducing trait-based task assignment approaches that exploit the agents' relative advantages. These approaches, however, assume that the agents' traits remain static. Indeed, in real-world scenarios, traits are likely to vary as agents execute tasks. In this paper, we present a transformation-based modeling framework to bridge the gap between state-of-the-art task assignment algorithms and the reality of dynamic traits. We define a transformation as a function that approximates dynamic traits with static traits based on a specific statistical measure. We define different candidate transformations, investigate their effects on different dynamic trait models, and the resulting task performance. Further, we propose a variance-based transformation as a general solution that approximates a variety of dynamic models, eliminating the need for hand specification. Finally, we demonstrate the benefits of reasoning about dynamic traits both in simulation and in a physical experiment involving the game of capture-the-flag.

ICRA Conference 2020 Conference Paper

Benchmark for Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance

Muhammad Asif Rana
Daphne Chen
Jacob Williams
Vivian Chu
Seyed Reza Ahmadzadeh
Sonia Chernova

We contribute a study benchmarking the performance of multiple motion-based learning from demonstration approaches. Given the number and diversity of existing methods, it is critical that comprehensive empirical studies be performed comparing the relative strengths of these techniques. In particular, we evaluate four approaches based on properties an end user may desire for real-world tasks. To perform this evaluation, we collected data from nine participants, across four manipulation tasks. The resulting demonstrations were used to train 180 task models and evaluated on 720 task reproductions on a physical robot. Our results detail how i) complexity of the task, ii) the expertise of the human demonstrator, and iii) the starting configuration of the robot affect task performance. The collected dataset of demonstrations, robot executions, and evaluations are publicly available. Research insights and guidelines are also provided to guide future research and deployment choices about these approaches.

ICRA Conference 2020 Conference Paper

CAGE: Context-Aware Grasping Engine

Weiyu Liu
Angel Andres Daruna
Sonia Chernova

Semantic grasping is the problem of selecting stable grasps that are functionally suitable for specific object manipulation tasks. In order for robots to effectively perform object manipulation, a broad sense of contexts, including object and task constraints, needs to be accounted for. We introduce the Context-Aware Grasping Engine, which combines a novel semantic representation of grasp contexts with a neural network structure based on the Wide & Deep model, capable of capturing complex reasoning patterns. We quantitatively validate our approach against three prior methods on a novel dataset consisting of 14, 000 semantic grasps for 44 objects, 7 tasks, and 6 different object states. Our approach outperformed all baselines by statistically significant margins, producing new insights into the importance of balancing memorization and generalization of contexts for semantic grasping. We further demonstrate the effectiveness of our approach on robot experiments in which the presented model successfully achieved 31 of 32 suitable grasps. The code and data are available at: https://github.com/wliu88/railsemanticgrasping.

ICRA Conference 2020 Conference Paper

Human-Centric Active Perception for Autonomous Observation

David Kent 0001
Sonia Chernova

As robot autonomy improves, robots are increasingly being considered in the role of autonomous observation systems - free-flying cameras capable of actively tracking human activity within some predefined area of interest. In this work, we formulate the autonomous observation problem through multi-objective optimization, presenting a novel Semi-MDP formulation of the autonomous human observation problem that maximizes observation rewards while accounting for both human- and robot-centric costs. We demonstrate that the problem can be solved with both scalarization-based Multi-Objective MDP methods and Constrained MDP methods, and discuss the relative benefits of each approach. We validate our work on activity tracking using a NASA Astrobee robot operating within a simulated International Space Station environment.

IROS Conference 2020 Conference Paper

Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging

Zackory Erickson
Eliot Xing
Bharat Srirangam
Sonia Chernova
Charles C. Kemp

Material recognition can help inform robots about how to properly interact with and manipulate real-world objects. In this paper, we present a multimodal sensing technique, leveraging near-infrared spectroscopy and close-range high resolution texture imaging, that enables robots to estimate the materials of household objects. We release a dataset of high resolution texture images and spectral measurements collected from a mobile manipulator that interacted with 144 house-hold objects. We then present a neural network architecture that learns a compact multimodal representation of spectral measurements and texture images. When generalizing material classification to new objects, we show that this multimodal representation enables a robot to recognize materials with greater performance as compared to prior state-of-the-art approaches. Finally, we present how a robot can combine this high resolution local sensing with images from the robot's head-mounted camera to achieve accurate material classification over a scene of objects on a table.

AAAI Conference 2020 Conference Paper

Path Ranking with Attention to Type Hierarchies

Weiyu Liu
Angel Daruna
Zsolt Kira
Sonia Chernova

The objective of the knowledge base completion problem is to infer missing information from existing facts in a knowledge base. Prior work has demonstrated the effectiveness of path-ranking based methods, which solve the problem by discovering observable patterns in knowledge graphs, consisting of nodes representing entities and edges representing relations. However, these patterns either lack accuracy because they rely solely on relations or cannot easily generalize due to the direct use of speciﬁc entity information. We introduce Attentive Path Ranking, a novel path pattern representation that leverages type hierarchies of entities to both avoid ambiguity and maintain generalization. Then, we present an end-to-end trained attention-based RNN model to discover the new path patterns from data. Experiments conducted on benchmark knowledge base completion datasets WN18RR and FB15k- 237 demonstrate that the proposed model outperforms existing methods on the fact prediction task by statistically signiﬁcant margins of 26% and 10%, respectively. Furthermore, quantitative and qualitative analyses show that the path patterns balance between generalization and discrimination.

JAAMAS Journal 2020 Journal Article

STRATA: unified framework for task assignments in large teams of heterogeneous agents

Harish Ravichandar
Kenneth Shaw
Sonia Chernova

Abstract Large teams of heterogeneous agents have the potential to solve complex multi-task problems that are intractable for a single agent working independently. However, solving complex multi-task problems requires leveraging the relative strengths of the different kinds of agents in the team. We present Stochastic TRAit-based Task Assignment (STRATA), a unified framework that models large teams of heterogeneous agents and performs effective task assignments. Specifically, given information on which traits (capabilities) are required for various tasks, STRATA computes the assignments of agents to tasks such that the trait requirements are achieved. Inspired by prior work in robot swarms and biodiversity, we categorize agents into different species (groups) based on their traits. We model each trait as a continuous variable and differentiate between traits that can and cannot be aggregated from different agents. STRATA is capable of reasoning about both species-level and agent-level variability in traits. Further, we define measures of diversity for any given team based on the team’s continuous-space trait model. We illustrate the necessity and effectiveness of STRATA using detailed experiments based in simulation and in a capture-the-flag game environment.

IJCAI Conference 2019 Conference Paper

Active Learning within Constrained Environments through Imitation of an Expert Questioner

Kalesha Bullard
Yannick Schroecker
Sonia Chernova

Active learning agents typically employ a query selection algorithm which solely considers the agent's learning objectives. However, this may be insufficient in more realistic human domains. This work uses imitation learning to enable an agent in a constrained environment to concurrently reason about both its internal learning goals and environmental constraints externally imposed, all within its objective function. Experiments are conducted on a concept learning task to test generalization of the proposed algorithm to different environmental conditions and analyze how time and resource constraints impact efficacy of solving the learning problem. Our findings show the environmentally-aware learning agent is able to statistically outperform all other active learners explored under most of the constrained conditions. A key implication is adaptation for active learning agents to more realistic human environments, where constraints are often externally imposed on the learner.

ICRA Conference 2019 Conference Paper

Real-time Multisensory Affordance-based Control for Adaptive Object Manipulation

Vivian Chu
Reymundo A. Gutierrez
Sonia Chernova
Andrea Thomaz

We address the challenge of how a robot can adapt its actions to successfully manipulate objects it has not previously encountered. We introduce Real-time Multisensory Affordance-based Control (RMAC), which enables a robot to adapt existing affordance models using multisensory inputs. We show that using the combination of haptic, audio, and visual information with RMAC allows the robot to learn afforance models and adaptively manipulate two very different objects (drawer, lamp), in multiple novel configurations. Offline evaluations and real-time online evaluations show that RMAC allows the robot to accurately open different drawer configurations and turn-on novel lamps with an average accuracy of 75%.

ICRA Conference 2019 Conference Paper

RoboCSE: Robot Common Sense Embedding

Angel Andres Daruna
Weiyu Liu
Zsolt Kira
Sonia Chernova

Autonomous service robots require computational frameworks that allow them to generalize knowledge to new situations in a manner that models uncertainty while scaling to real-world problem sizes. The Robot Common Sense Embedding (RoboCSE) showcases a class of computational frameworks, multi-relational embeddings, that have not been leveraged in robotics to model semantic knowledge. We validate RoboCSE on a realistic home environment simulator (AI2Thor) to measure how well it generalizes learned knowledge about object affordances, locations, and materials. Our experiments show that RoboCSE can perform prediction better than a baseline that uses pre-trained embeddings, such as Word2Vec, achieving statistically significant improvements while using orders of magnitude less memory than our Bayesian Logic Network baseline. In addition, we show that predictions made by RoboCSE are robust to significant reductions in data available for training as well as domain transfer to MatterPort3D, achieving statistically significant improvements over a baseline that memorizes training data.

ICRA Conference 2019 Conference Paper

Skill Acquisition via Automated Multi-Coordinate Cost Balancing

Harish Ravichandar
Seyed Reza Ahmadzadeh
Muhammad Asif Rana
Sonia Chernova

We propose a learning framework, named Multi-Coordinate Cost Balancing (MCCB), to address the problem of acquiring point-to-point movement skills from demonstrations. MCCB encodes demonstrations simultaneously in multiple differential coordinates that specify local geometric properties. MCCB generates reproductions by solving a convex optimization problem with a multi-coordinate cost function and linear constraints on the reproductions, such as initial, target, and via points. Further, since the relative importance of each coordinate system in the cost function might be unknown for a given skill, MCCB learns optimal weighting factors that balance the cost function. We demonstrate the effectiveness of MCCB via detailed experiments conducted on one handwriting dataset and three complex skill datasets.

ICRA Conference 2019 Conference Paper

Tool Macgyvering: Tool Construction Using Geometric Reasoning

Lakshmi Nair 0001
Jonathan C. Balloch
Sonia Chernova

MacGyvering is defined as creating or repairing something in an inventive or improvised way by utilizing objects that are available at hand. In this paper, we explore a subset of Macgyvering problems involving tool construction, i. e. , creating tools from parts available in the environment. We formalize the overall problem domain of tool Macgyvering, introducing three levels of complexity for tool construction and substitution problems, and presenting a novel computational framework aimed at solving one level of the tool Macgyvering problem, specifically contributing a novel algorithm for tool construction based on geometric reasoning. We validate our approach by constructing three tools using a 7-DOF robot arm.

AAMAS Conference 2019 Conference Paper

Unsupervised Role Discovery Using Temporal Observations of Agents

Andrew Silva
Sonia Chernova

Agent-based modeling of multi-agent systems has enormous potential with applications in modeling social, economic, medical and other application domains containing temporal data. We propose an unsupervised approach to discovering common roles by observing agents over time, allowing us to construct a role-based representation of multi-agent systems that aids in understanding and interpreting the state of the system. We validate our approach on both a soccer and a StarCraft dataset, and show that unsupervised role discovery through observation can provide meaningful insight into the state of a multi-agent system, aiding or even replacing game state data for interpretation or understanding of the system.

AAMAS Conference 2018 Conference Paper

Action Categorization for Computationally Improved Task Learning and Planning

Lakshmi Nair
Sonia Chernova

This paper explores the problem of task learning and planning, contributing the Action-Category Representation (ACR) to improve computational performance of both Planning and Reinforcement Learning (RL). ACR is an algorithm-agnostic, abstract data representation that maps objects to action categories (groups of actions), inspired by the psychological concept of action codes. We validate our approach in StarCraft and Lightworld domains; our results demonstrate several benefits of ACR relating to improved computational performance of planning and RL, by reducing the action space for the agent.

ICRA Conference 2018 Conference Paper

Human-Driven Feature Selection for a Robotic Agent Learning Classification Tasks from Demonstration

Kalesha Bullard
Sonia Chernova
Andrea Thomaz

The state features available to a robot define the variables on which the learning computation depends. However, little prior work considers feature selection in the context of deploying a general-purpose robot able to learn new tasks. In this work, we explore human-driven feature selection in which a robotic agent can identify useful features with the aid of a human user, by extracting information from users about which features are most informative for discriminating between classes of objects needed for a given task (e. g. sorting groceries). The research questions examine (a) whether a domain expert is able to identify a subset of informative task features, (b) whether human selected features will enable the agent to classify unseen examples as accurately as using computational feature selection, and (c) if the interaction strategy used to elicit the information from the user impacts the quality of resulting feature selection. Toward that end, we conducted a user study with 30 participants on campus, given a multi-class classification task and one of five different approaches for conveying information about informative features to a robot learner. Our findings show that when features are semantically interpretable, human feature selection is effective in LfD scenarios because it is able to outperform computational methods when there is limited training data, yet still remains on-par with computational methods as the training sample size increases.

IROS Conference 2018 Conference Paper

Learning Generalizable Robot Skills from Demonstrations in Cluttered Environments

Muhammad Asif Rana
Mustafa Mukadam
Seyed Reza Ahmadzadeh
Sonia Chernova
Byron Boots

Learning from Demonstration (LfD) is a popular approach to endowing robots with skills without having to program them by hand. Typically, LfD relies on human demonstrations in clutter-free environments. This prevents the demonstrations from being affected by irrelevant objects, whose influence can obfuscate the true intention of the human or the constraints of the desired skill. However, it is unrealistic to assume that the robot's environment can always be restructured to remove clutter when capturing human demonstrations. To contend with this problem, we develop an importance weighted batch and incremental skill learning approach, building on a recent inference-based technique for skill representation and reproduction. Our approach reduces unwanted environmental influences on the learned skill, while still capturing the salient human behavior. We provide both batch and incremental versions of our approach and validate our algorithms on a 7-DOF JACO2 manipulator with reaching and placing skills.

IROS Conference 2018 Conference Paper

Towards Intelligent Arbitration of Diverse Active Learning Queries

Kalesha Bullard
Andrea Thomaz
Sonia Chernova

Active learning literature has explored the selection of optimal queries by a learning agent with respect to given criteria, but prior work in classification has focused only on obtaining labels for queried samples. In contrast, proficient learners, like humans, integrate multiple forms of information during learning. This work seeks to enable an active learner to reason about multiple query types concurrently, aimed at soliciting both instance and feature information from the teacher, and to autonomously arbitrate between queries of different types. We contribute the design of rule-based and decision-theoretic arbitration strategies and evaluate all against baselines of more traditional passive and active learning. Our findings show that all arbitration strategies lead to more efficient learning, compared to the baselines. Moreover, given a dynamically changing environment and constrained questioning budget (typical in human settings), the decision-theoretic strategy statistically outperforms all other methods since it reasons about both what query to make and when to make a query, in order to most effectively utilize its questioning budget.

AAMAS Conference 2017 Conference Paper

Temporal Models for Robot Classification of Human Interruptibility

Siddhartha Banerjee
Sonia Chernova

Robots are increasingly being deployed in unstructured human environments where they will need to approach and interrupt collocated humans. Most prior work on robot interruptions has focused on how to interrupt a person or on estimating a human’s awareness of the robot. Our work makes three contributions to this research area. First, we introduce an ordinal scale of interruptibility that can be used to rate the interruptibility of a human. Second, we propose the use of Conditional Random Fields (CRFs) and their variants, Hidden CRFs, and Latent-Dynamic CRFs, for classifying interruptibility. Third, we introduce the use of object labels as a visual cue to the context of an interruption in order to improve interruptibility estimates. Our results show that Latent-Dynamic CRFs outperform all other models across all tested conditions, and that the inclusion of object labels as a cue to context improves interruptibility classification performance, yielding the best overall results.

ICRA Conference 2017 Conference Paper

Temporal persistence modeling for object search

Russell Toris
Sonia Chernova

We present a novel solution to the object search problem for domains in which object permanence cannot be assumed and other agents may move objects between locations without the robot's knowledge. We formalize object search as a failure analysis problem and contribute temporal persistence modeling (TPM), an algorithm for probabilistic prediction of the time that an object is expected to remain at a given location given sparse prior observations. We show that probabilistic exponential distributions augmented with a Gaussian component can accurately represent probable object locations and search suggestions based entirely on sparsely made visual observations. We evaluate our work in two domains, a large scale GPS location data set for person tracking, and multi-object tracking on a mobile robot operating in a small-scale household environment over a 2-week period. TPM performance exceeds four baseline methods across all study conditions.

AAMAS Conference 2016 Conference Paper

Learning from Demonstration for Shaping through Inverse Reinforcement Learning

Halit Bener Suay
Tim Brys
Matthew E. Taylor
Sonia Chernova

Model-free episodic reinforcement learning problems define the environment reward with functions that often provide only sparse information throughout the task. Consequently, agents are not given enough feedback about the fitness of their actions until the task ends with success or failure. Previous work addresses this problem with reward shaping. In this paper we introduce a novel approach to improve modelfree reinforcement learning agents’ performance with a three step approach. Specifically, we collect demonstration data, use the data to recover a linear function using inverse reinforcement learning and we use the recovered function for potential-based reward shaping. Our approach is model-free and scalable to high dimensional domains. To show the scalability of our approach we present two sets of experiments in a two dimensional Maze domain, and the 27 dimensional Mario AI domain. We compare the performance of our algorithm to previously introduced reinforcement learning from demonstration algorithms. Our experiments show that our approach outperforms the state-of-the-art in cumulative reward, learning rate and asymptotic performance.

IJCAI Conference 2015 Conference Paper

Reinforcement Learning from Demonstration through Shaping

Tim Brys
Anna Harutyunyan
Halit Bener Suay
Sonia Chernova
Matthew E. Taylor
Ann Now
eacute;

Reinforcement learning describes how a learning agent can achieve optimal behaviour based on interactions with its environment and reward feedback. A limiting factor in reinforcement learning as employed in artificial intelligence is the need for an often prohibitively large number of environment samples before the agent reaches a desirable level of performance. Learning from demonstration is an approach that provides the agent with demonstrations by a supposed expert, from which it should derive suitable behaviour. Yet, one of the challenges of learning from demonstration is that no guarantees can be provided for the quality of the demonstrations, and thus the learned behavior. In this paper, we investigate the intersection of these two approaches, leveraging the theoretical guarantees provided by reinforcement learning, and using expert demonstrations to speed up this learning by biasing exploration through a process called reward shaping. This approach allows us to leverage human input without making an erroneous assumption regarding demonstration optimality. We show experimentally that this approach requires significantly fewer demonstrations, is more robust against suboptimality of demonstrations, and achieves much faster learning than the recently developed HAT algorithm.

RLDM Conference 2015 Conference Abstract

Reward Shaping by Demonstration

Halit Suay
Sonia Chernova
Tim Brys
Vrije Universiteit Brussel
Matthew Taylor

Potential-based reward shaping is a theoretically sound way of incorporating prior knowledge in a reinforcement learning setting. While providing flexibility for choosing the potential function, this method guarantees the convergence of the final policy, regardless of the properties of the potential function. How- ever, this flexibility of choice, may cause confusion when making a design decision for a specific domain, as the number of possible candidates for a potential function can be overwhelming. Moreover, the poten- tial function either can be manually designed, to bias the behavior of the learner, or can be recovered from prior knowledge, e. g. from human demonstrations. In this paper we investigate the efficacy of two different ways for using a potential function recovered from human demonstrations. First approach uses a mixture of Gaussian distributions generated by samples collected during demonstrations (Gaussian-Shaping), and the second approach uses a reward function recovered from demonstrations with Relative Entropy Inverse Re- inforcement Learning (RE-IRL-Shaping). We present our findings in Cart-Pole, Mountain Car, and Puddle World domains. Our results show that Gaussian-Shaping can provide an efficient reward heuristic, acceler- ating learning through its ability to capture local information, and RE-IRL-Shaping can be more resilient to bad demonstrations. We report a brief analysis of our findings and we aim to provide a future reference for reinforcement learning agent designers, who consider using reward shaping by human demonstrations.

IROS Conference 2015 Conference Paper

Robot Web Tools: Efficient messaging for cloud robotics

Russell Toris
Julius Kammerl
David V. Lu
Jihoon Lee
Odest Chadwicke Jenkins
Sarah Osentoski
Mitchell Wills
Sonia Chernova

Since its official introduction in 2012, the Robot Web Tools project has grown tremendously as an open-source community, enabling new levels of interoperability and portability across heterogeneous robot systems, devices, and front-end user interfaces. At the heart of Robot Web Tools is the rosbridge protocol as a general means for messaging ROS topics in a client-server paradigm suitable for wide area networks, and human-robot interaction at a global scale through modern web browsers. Building from rosbridge, this paper describes our efforts with Robot Web Tools to advance: 1) human-robot interaction through usable client and visualization libraries for more efficient development of front-end human-robot interfaces, and 2) cloud robotics through more efficient methods of transporting high-bandwidth topics (e. g. , kinematic transforms, image streams, and point clouds). We further discuss the significant impact of Robot Web Tools through a diverse set of use cases that showcase the importance of a generic messaging protocol and front-end development systems for human-robot interaction.

AAAI Conference 2015 Conference Paper

Solving and Explaining Analogy Questions Using Semantic Networks

Adrian Boteanu
Sonia Chernova

Analogies are a fundamental human reasoning pattern that relies on relational similarity. Understanding how analogies are formed facilitates the transfer of knowledge between contexts. The approach presented in this work focuses on obtaining precise interpretations of analogies. We leverage noisy semantic networks to answer and explain a wide spectrum of analogy questions. The core of our contribution, the Semantic Similarity Engine, consists of methods for extracting and comparing graph-contexts that reveal the relational parallelism that analogies are based on, while mitigating uncertainty in the semantic network. We demonstrate these methods in two tasks: answering multiple choice analogy questions and generating human readable analogy explanations. We evaluate our approach on two datasets totaling 600 analogy questions. Our results show reliable performance and low false-positive rate in question answering; human evaluators agreed with 96% of our analogy explanations.

ICRA Conference 2015 Conference Paper

Unsupervised learning of multi-hypothesized pick-and-place task templates via crowdsourcing

Russell Toris
David Kent 0001
Sonia Chernova

In order for robots to be useful in real world learning scenarios, non-expert human teachers must be able to interact with and teach robots in an intuitive manner. One essential robot capability is wide-area (mobile or nonstationary) pick-and-place tasks. Even in its simplest form, pick-and-place is a hard problem due to uncertainty arising from noisy input demonstrations and non-deterministic real world environments. This work introduces a novel method for goal-based learning from demonstration where we learn over a large corpus of human demonstrated ground truths of placement locations in an unsupervised manner via Gaussian Mixture Models. The goal is to provide a multi-hypothesis solution for a given task description which can later be utilized in the execution of the task itself. In addition to learning the actual arrangements of the items in question, we also autonomously extract which frames of reference are important in each demonstration. We further verify these findings in a subsequent evaluation and execution via a mobile manipulator.

IROS Conference 2014 Conference Paper

Construction of an object manipulation database from grasp demonstrations

David Kent 0001
Sonia Chernova

Intelligent object manipulation is critical for a robot to effectively operate in a household environment. There are many grasp planners that can estimate grasps based on object shape, but these approaches often perform poorly because they miss key information about non-visual object characteristics. Object model databases can account for this information, but existing methods for database construction are time and resource intensive. We present an easy-to-use system for constructing a grasp database from crowdsourced demonstrations. The method requires no additional equipment other than the robot itself, and non-expert users can demonstrate grasps through an intuitive web interface, with virtually no training required. We show that the crowdsourced grasps can prove sufficient for object manipulation, and furthermore the demonstration approach outperforms purely vision-based grasp planning approaches for a wide variety of object classes.

ICRA Conference 2014 Conference Paper

Crowdsourcing the construction of a 3D object recognition database for robotic grasping

David Kent 0001
Morteza Behrooz
Sonia Chernova

Object recognition and manipulation are critical in enabling robots to interact with objects in a household environment. Construction of 3D object recognition databases is time and resource intensive, often requiring specialized equipment, and is therefore difficult to apply to robots in the field. We present a system for constructing object models for 3D object recognition and manipulation made possible by advances in web robotics. The database consists of point clouds generated using a novel iterative point cloud registration algorithm, which includes the potential to encode manipulation data and usability characteristics. We validate the system with a crowdsourcing user study and object recognition system designed to work with our object recognition database.

ICRA Conference 2014 Conference Paper

DARPA Robotics Challenge: Towards a user-guided manipulation framework for high-DOF robots

Nicholas Alunni
Halit Bener Suay
Calder Phillips-Grafflin
Jim Mainprice
Dmitry Berenson
Sonia Chernova
Robert W. Lindeman
Daniel M. Lofaro

Supervision and teleoperation of high degree-of-freedom robots is a complex task due to environmental constraints such as obstacles and limited communication, as well as task specific requirements such as using more than one end-effector at the same time. In this work we present a supervision and teleoperation framework that allows an operator to see the surroundings of a robot in 3D, make necessary adjustments for a dual or single arm manipulation task, preview the task in simulation before execution, and finally execute the task on a real robot. The framework has been applied to the valve turning task of the DARPA Robotics Challenge on the PR2, Hubo2+, and DRCHubo robots.

IROS Conference 2014 Conference Paper

From autonomy to cooperative traded control of humanoid manipulation tasks with unreliable communication: System design and lessons learned

Jim Mainprice
Calder Phillips-Grafflin
Halit Bener Suay
Nicholas Alunni
Daniel M. Lofaro
Dmitry Berenson
Sonia Chernova
Robert W. Lindeman

In this paper, we report lessons learned through the design of a framework for teleoperating a humanoid robot to perform a manipulation task. We present a software framework for cooperative traded control that enables a team of operators to control a remote humanoid robot over an unreliable communications link. The framework produces statically-stable motion trajectories that are collision-free and respect end-effector pose constraints. After operator confirmation, these trajectories are sent over the data link for execution on the robot. Additionally, we have defined a clear operational procedure for the operators to manage the teleoperation task. We applied our system to the valve turning task in the DARPA Robotics Challenge (DRC). Our framework is able to perform reliably and is resilient to unreliable network conditions, as we demonstrate in a set of test runs performed remotely over the internet. We analyze our approach and discuss lessons learned which may be useful for others when designing such a system.

AAMAS Conference 2011 Conference Paper

Integrating Reinforcement Learning with Human Demonstrations of Varying Ability

Matthew E. Taylor
Halit Bener Suay
Sonia Chernova

This work introduces Human-Agent Transfer (HAT), an algorithm that combines transfer learning, learning from demonstration and reinforcement learning to achieve rapid learning and high performance in complex domains. Using experiments in a simulated robot soccer domain, we show that human demonstrations transferred into a baseline policy for an agent and refined using reinforcement learning significantly improve both learning time and policy performance. Our evaluation compares three algorithmic approaches to incorporating demonstration rule summaries into transfer learning, and studies the impact of demonstration quality and quantity, as well as the effect of combining demonstrations from multiple teachers. Our results show that all three transfer methods lead to statistically significant improvement in performance over learning without demonstration. The best performance was achieved by combining the best demonstrations from two teachers.

IROS Conference 2008 Conference Paper

Learning equivalent action choices from demonstration

Sonia Chernova
Manuela Veloso

In their interactions with the world robots inevitably face equivalent action choices, situations in which multiple actions are equivalently applicable. In this paper, we address the problem of equivalent action choices in learning from demonstration, a robot learning approach in which a policy is acquired from human demonstrations of the desired behavior. We note that when faced with a choice of equivalent actions, a human teacher often demonstrates an action arbitrarily and does not make the choice consistently over time. The resulting inconsistently labeled training data poses a problem for classification-based demonstration learning algorithms by violating the common assumption that for any world state there exists a single best action. This problem has been overlooked by previous approaches for demonstration learning. In this paper, we present an algorithm that identifies regions of the state space with conflicting demonstrations and enables the choice between multiple actions to be represented explicitly within the robotpsilas policy. An experimental evaluation of the algorithm in a real-world obstacle avoidance domain shows that reasoning about action choices significantly improves the robotpsilas learning performance.

AAMAS Conference 2008 Conference Paper

Teaching Multi-Robot Coordination using Demonstration of Communication and State Sharing

Sonia Chernova
Manuela Veloso

Solutions to complex tasks often require the cooperation of multiple robots, however, developing multi-robot policies can present many challenges. In this work, we introduce teaching by demonstration in the context of multi-robot tasks, enabling a single teacher to instruct multiple robots to work together through a demonstration of the desired behavior. Within this framework, we contribute two approaches for teaching coordination based on different communication and information sharing strategies. To enable the teacher to divide attention between multiple robots, each robot uses a confidence-based algorithm that allows it to regulate its autonomy and determine the need for demonstration. Evaluation is performed using two Sony QRIO robots learning a real-world collaborative ball sorting task.

AAMAS Conference 2007 Conference Paper

Confidence-Based Policy Learning from Demonstration Using Gaussian Mixture Models

Sonia Chernova
Manuela Veloso

We contribute an approach for interactive policy learning through expert demonstration that allows an agent to actively request and effectively represent demonstration examples. In order to address the inherent uncertainty of human demonstration, we represent the policy as a set of Gaussian mixture models (GMMs), where each model, with multiple Gaussian components, corresponds to a single action. Incrementally received demonstration examples are used as training data for the GMM set. We then introduce our confident execution approach, which focuses learning on relevant parts of the domain by enabling the agent to identify the need for and request demonstrations for specific parts of the state space. The agent selects between demonstration and autonomous execution based on statistical analysis of the uncertainty of the learned Gaussian mixture set. As it achieves proficiency at its task and gains confidence in its actions, the agent operates with increasing autonomy, eliminating the need for unnecessary demonstrations of already acquired behavior, and reducing both the training time and the demonstration workload of the expert. We validate our approach with experiments in simulated and real robot domains.

IROS Conference 2004 Conference Paper

An evolutionary approach to gait learning for four-legged robots

Sonia Chernova
Manuela Veloso

Developing fast gaits for legged robots is a difficult task that requires optimizing parameters in a highly irregular, multidimensional space. In the past, walk optimization for quadruped robots, namely the Sony AIBO robot, was done by handtuning the parameterized gaits. In addition to requiring a lot of time and human expertise, this process produced sub-optimal results. Several recent projects have focused on using machine learning to automate the parameter search. Algorithms utilizing Powell's minimization method and policy gradient reinforcement learning have shown significant improvement over previous walk optimization results. In this paper we present a new algorithm for walk optimization based on an evolutionary approach. Unlike previous methods, our algorithm does not attempt to approximate the gradient of the multidimensional space. This makes it more robust to noise in parameter evaluations and avoids prematurely converging to local optima, a problem encountered by both of the previously suggested algorithms. Our evolutionary algorithm matches the best previous learning method, achieving several different walks of high quality. Furthermore, the best learned walks represent an impressive 20% improvement over our own best hand-tuned walks.

ICRA Conference 2004 Conference Paper

Learning and using Models of Kicking Motions for Legged Robots

Sonia Chernova
Manuela Veloso

Legged robots, such as the Sony AIBO, create opportunity to design rich motions to be executed in specific situations. In particular, teams involved in robot soccer RoboCup competitions have developed many different motions for kicking the ball. Designing effective motions and determining their effects is a challenging problem that is traditionally approached through a generate and test methodology. In this paper, we present a method we developed for learning the effects of kicking motions. Our procedure acquires models of the kicks in terms of key values that describe their effects on the ball's trajectory, namely the angle and the distance reached. The successful automated acquisition of the models of different kicks is then followed by the incorporation of these models into the behaviors to select the most promising kick in a given state of the world. Using the robot soccer domain, we demonstrate that a robot that takes into account the learned predicted effects of its actions performs significantly better than its counterpart.