Aaron F. Bobick

ICRA Conference 2016 Conference Paper

Affordance-feasible planning with manipulator wrench spaces

• Andrew Price
• Stephen Balakirsky
• Aaron F. Bobick
• Henrik I. Christensen

This work introduces an affordance characterization employing mechanical wrenches as a metric for predicting and planning with workspace affordances. Although affordances are a commonly used high-level paradigm for robotic task-level planning and learning, the literature has been sparse regarding how to characterize the agent in this object-agent-environment framework. In this work, we propose decomposing a behavior into a vocabulary of characteristic requirements and capabilities that are suitable to predict the affordances of various parts of the workspace. Specifically, we investigate mechanical wrenches as a viable representation of these affordance requirements and capabilities. We then use this vocabulary in a planning system to compose complex motions from simple behavior types in continuous space. The utility of the framework for complex planning is demonstrated on example scenarios both in simulation and with real-world industrial manipulators.

ICRA Conference 2015 Conference Paper

Augmenting physical state prediction through structured activity inference

• Nam N. Vo
• Aaron F. Bobick

We address the problem of predicting the physical state of a an agent performing a known activity. In particular we are interested in predicting human movement during complex composite activities. Our proposed framework combines a graphical model that extends the Sequential Interval Network (SIN) [1] for modeling global temporal structure of activities with a low level dynamic system for modeling the dynamics of the physical state. Specifically, two sets of new hidden state variables are added: one with respect to the temporal structure and one with respect to time. A mapping factor is defined to ensure these variables values remain consistent and hence allows fusing the two sources of information. We then derive an inference algorithm for computing the posterior densities of the hidden variables. The system can run in an on-line predictive mode to recognize on-going activity and make predictions arbitrarily far in the future during execution of the activity. Experiments illustrate that the long term prediction performance benefits from the knowledge about the temporal structure of the activity while short term prediction performance is improved by incorporating the dynamics of physical state.

ICRA Conference 2014 Conference Paper

Anticipating human actions for collaboration in the presence of task and sensor uncertainty

• Kelsey P. Hawkins
• Shray Bansal
• Nam N. Vo
• Aaron F. Bobick

A representation for structured activities is developed that allows a robot to probabilistically infer which task actions a human is currently performing and to predict which future actions will be executed and when they will occur. The goal is to enable a robot to anticipate collaborative actions in the presence of uncertain sensing and task ambiguity. The system can represent multi-path tasks where the task variations may contain partially ordered actions or even optional actions that may be skipped altogether. The task is represented by an AND-OR tree structure from which a probabilistic graphical model is constructed. Inference methods for that model are derived that support a planning and execution system for the robot which attempts to minimize a cost function based upon expected human idle time. We demonstrate the theory in both simulation and actual human-robot performance of a two-way-branch assembly task. In particular we show that the inference model can robustly anticipate the actions of the human even in the presence of unreliable or noisy detections because of its integration of all its sensing information along with knowledge of task structure.

ICRA Conference 2013 Conference Paper

Decoupling behavior, perception, and control for autonomous learning of affordances

• Tucker Hermans
• James M. Rehg
• Aaron F. Bobick

A novel behavior representation is introduced that permits a robot to systematically explore the best methods by which to successfully execute an affordance-based behavior for a particular object. The approach decomposes affordance-based behaviors into three components. We first define controllers that specify how to achieve a desired change in object state through changes in the agent's state. For each controller we develop at least one behavior primitive that determines how the controller outputs translate to specific movements of the agent. Additionally we provide multiple perceptual proxies that define the representation of the object that is to be computed as input to the controller during execution. A variety of proxies may be selected for a given controller and a given proxy may provide input for more than one controller. When developing an appropriate affordance-based behavior strategy for a given object, the robot can systematically vary these elements as well as note the impact of additional task variables such as location in the workspace. We demonstrate the approach using a PR2 robot that explores different combinations of controller, behavior primitive, and proxy to perform a push or pull positioning behavior on a selection of household objects, learning which methods best work for each object.

IROS Conference 2012 Conference Paper

Guided pushing for object singulation

• Tucker Hermans
• James M. Rehg
• Aaron F. Bobick

We propose a novel method for a robot to separate and segment objects in a cluttered tabletop environment. The method leverages the fact that external object boundaries produce visible edges within an object cluster. We achieve this singulation of objects by using the robot arm to perform pushing actions specifically selected to test whether particular visible edges correspond to object boundaries. We verify the separation of objects after a push by examining the clusters formed by geometric segmentation of regions residing on the table surface. To avoid explicitly representing and tracking edges across push behaviors we aggregate over all edges in a given orientation by representing the push-history as an orientation histogram. By tracking the history of directions pushed for each object cluster we can build evidence that a cluster cannot be further separated. We present quantitative and qualitative experimental results performed in a real home environment by a mobile manipulator using input from an RGB-D camera mounted on the robot's head. We show that our pushing strategy can more reliably obtain singulation in fewer pushes than an approach, that does not explicitly reason about boundary information.

IROS Conference 2012 Conference Paper

Learning a projective mapping to locate animals in video using RFID

• Pipei Huang
• Rahul Sawhney
• Daniel Walker
• Kim Wallen
• Aaron F. Bobick
• Shiyin Qin
• Tucker R. Balch

We present a method to locate animals in video based on their reported positions using noisy and biased measurements from a radio frequency identification (RFID) system. The system uses a kernel regression method to learn a mapping from reported X, Y, Z locations in the environment to X, Y pixel locations in video with minimal calibration and training data. Our goal is for this system to facilitate animal behavior research by enabling automatic identification of interactions between animals and then providing the location of the animals in video so that the details of each interaction can be examined more closely by either humans or machines. The primary contribution of this work is achieving efficient and reliable 3D to 2D projective mapping in a non-parametric way while also overcoming challenges that would otherwise affect accuracy. Our system successfully addresses issues regarding noisy positional data, position bias, occlusion of RFID tags, and wide angle lens distortion. We validate the system experimentally indoors as well as in the field and compare the accuracy of our system with the standard camera projection model-based procedure.

IROS Conference 2009 Conference Paper

Effective Robot Task Learning by focusing on Task-relevant objects

• Kyuhwa Lee
• Jinhan Lee
• Andrea Thomaz
• Aaron F. Bobick

In a Robot Learning from Demonstration framework involving environments with many objects, one of the key problems is to decide which objects are relevant to a given task. In this paper, we analyze this problem and propose a biologically-inspired computational model that enables the robot to focus on the task-relevant objects. To filter out incompatible task models, we compute a Task Relevance Value (TRV) for each object, which shows a human demonstrator's implicit indication of the relevance to the task. By combining an intentional action representation with ‘motionese’ [2], our model exhibits recognition capabilities compatible with the way that humans demonstrate. We evaluate the system on demonstrations from five different human subjects, showing its ability to correctly focus on the appropriate objects in these demonstrations.

ICRA Conference 2006 Conference Paper

Traversability Classification using Unsupervised on-line Visual Learning for Outdoor Robot Navigation

• Dongshin Kim 0002
• Jie Sun 0004
• Sang Min Oh
• James M. Rehg
• Aaron F. Bobick

Estimating the traversability of terrain in an unstructured outdoor environment is a core functionality for autonomous robot navigation. While general-purpose sensing can be used to identify the existence of terrain features such as vegetation and sloping ground, the traversability of these regions is a complex function of the terrain characteristics and vehicle capabilities, which makes it extremely difficult to characterize a priori. Moreover, it is difficult to find general rules which work for a wide variety of terrain types such as trees, rocks, tall grass, logs, and bushes. As a result, methods which provide traversability estimates based on predefined terrain properties such as height or shape will be unlikely to work reliably in unknown outdoor environments. Our approach is based on the observation that traversability in the most general sense is an affordance which is jointly determined by the vehicle and its environment. We describe a novel on-line learning method which can make accurate predictions of the traversability properties of complex terrain. Our method is based on autonomous training data collection which exploits the robot's experience in navigating its environment to train classifiers without human intervention. This is in contrast to other learning methods in which training data is collected manually. We have implemented and tested our traversability learning method on an unmanned ground vehicle (UGV) and evaluated its performance in several realistic outdoor environments. The experiments quantify the benefit of our on-line traversability learning approach

UAI Conference 2005 Conference Paper

Unsupervised Activity Discovery and Characterization From Event-Streams

• Rafay Hammid
• Siddhartha Maddi
• Amos Y. Johnson
• Aaron F. Bobick
• Irfan Essa
• Charles Isbell

AAAI Conference 1999 Conference Paper

A Framework for Recognizing Multi-Agent Action from Visual Evidence

• Stephen S. Intille
• Aaron F. Bobick
• MIT Media Laboratory

A probabilistic framework forrepresenting and visually recognizing complex multi-agent action is presented. Motivated by work in model-based object recognition and designed for the recognition of action from visual evidence, the representation has three components: (1) temporal structure descriptions representing the temporal relationships between agent goals, (2) belief networks for probabilistically representing and recognizing individual agentgoals from visualevidence, and (3) belief networks automatically generated from the temporalstructure descriptions that supportthe recognition of the complex action. We describe our current work on recognizing American football plays from noisy trajectory data. 1

ICRA Conference 1989 Conference Paper

Exploiting temporal coherence in scene analysis for autonomous navigation

• Robert C. Bolles
• Aaron F. Bobick

A technique for building reliable scene descriptions by evaluating the temporal stability of detected objects is presented. The approach is designed to avoid mistakes and to increase the competence of the sensing system by tracking objects from image to image and evaluating the stability of their descriptions over time. Since the information available about an object can change significantly over time, the authors introduce the idea of a representation space, which is a lattice of representations progressing from crude blob descriptions to complete semantic models, such as bush, rock, and tree. One of these representations is associated with an object only after the object has been described multiple times in the representation and the parameters of the representation are stable in a statistical sense enhanced by a set of explanations describing valid reasons for deviations. To illustrate the power of these ideas, the authors have implemented a system, called TraX, that constructs and refines models of outdoor objects detected in sequences of range data. >