Dana H. Ballard

IJCAI Conference 2019 Conference Paper

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

• Ruohan Zhang
• Faraz Torabi
• Lin Guan
• Dana H. Ballard
• Peter Stone

Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment. Human knowledge of how to solve these tasks can be incorporated using imitation learning, where the agent learns to imitate human demonstrated decisions. However, human guidance is not limited to the demonstrations. Other types of guidance could be more suitable for certain tasks and require less human effort. This survey provides a high-level overview of five recent learning frameworks that primarily rely on human guidance other than conventional, step-by-step action demonstrations. We review the motivation, assumption, and implementation of each framework. We then discuss possible future research directions.

IJCAI Conference 2016 Conference Paper

Decision-Making Policies for Heterogeneous Autonomous Multi-Agent Systems with Safety Constraints

• Ruohan Zhang
• Yue Yu
• Mahmoud El Chamie
• Beh
• ccedil; et A
• ccedil; ıkmese
• Dana H. Ballard

This paper studies a decision-making problem for heterogeneous multi-agent systems with safety density constraints. An individual agent's decision-making problem is modeled by the standard Markov Decision Process (MDP) formulation. However, an important special case occurs when the MDP states may have limited capacities, hence upper bounds on the expected number of agents in each state are imposed. We refer to these upper bound constraints as "safety" constraints. If agents follow unconstrained policies (policies that do not impose the safety constraints), the safety constraints might be violated. In this paper, we devise algorithms that provide safe decision-making policies. The set of safe decision policies can be shown to be convex, and hence the policy synthesis is tractable via reliable and fast Interior Point Method (IPM) algorithms. We evaluate the effectiveness of proposed algorithms first using a simple MDP, and then using a dynamic traffic assignment problem. The numerical results demonstrate that safe decision-making algorithms in this paper significantly outperform other baselines.

IROS Conference 2011 Conference Paper

Novelty detection using Growing Neural Gas for visuo-spatial memory

• Dmitry Kit
• Brian T. Sullivan
• Dana H. Ballard

Detecting visual changes in environments is an important computation with many applications in robotics and computer vision. Security cameras, remotely operated vehicles, and sentry robots could all benefit from robust change detection capability. We conjecture that if one has a mobile camera system the number of visual scenes that are experienced is limited (compared to the space of all possible scenes) and that the scenes do not frequently undergo major changes between observations. These assumptions can be exploited to ease the task of change detection and reduce the computational complexity of processing visual information by utilizing memory to store previous computations.

IROS Conference 2006 Conference Paper

Motor Synergies for Coordinated Movements in Humanoids

• Xue Gu
• Dana H. Ballard

Synthesizing automatons whole body movements is difficult, especially in humanoids with as high degrees of freedoms (DOFs) as humans. Based on a biologically inspired control model we proposed, three different ways of motor synergies over multiple motor routines are discussed to compose complex movements in a 33 DOF humanoid. Motor routine is a movement unit which implements a functional task and only involves those active joints participating in the task. We demonstrate the humanoid doing coordinated walking, sitting and rising, reaching, object manipulation etc

AAAI Conference 2004 Conference Paper

On the Integration of Grounding Language and Learning Objects

• Chen Yu
• Dana H. Ballard

This paper presents a multimodal learning system that can ground spoken names of objects in their physical referents and learn to recognize those objects simultaneously from naturally co-occurring multisensory input. There are two technical problems involved: (1) the correspondence problem in symbol grounding – how to associate words (symbols) with their perceptually grounded meanings from multiple cooccurrences between words and objects in the physical environment. (2) object learning – how to recognize and categorize visual objects. We argue that those two problems can be fundamentally simplified by considering them in a general system and incorporating the spatio-temporal and crossmodal constraints of multimodal data. The system collects egocentric data including image sequences as well as speech while users perform natural tasks. It is able to automatically infer the meanings of object names from vision, and categorize objects based on teaching signals potentially encoded in speech. The experimental results reported in this paper reveal the effectiveness of using multimodal data and integrating heterogeneous techniques in machine learning, natural language processing and computer vision.

AIJ Journal 1995 Journal Article

An active vision architecture based on iconic representations

• Rajesh P.N. Rao
• Dana H. Ballard

Active vision systems have the capability of continuously interacting with the environment. The rapidly changing environment of such systems means that it is attractive to replace static representations with visual routines that compute information on demand. Such routines place a premium on image data structures that are easily computed and used. The purpose of this paper is to propose a general active vision architecture based on efficiently computable iconic representations. This architecture employs two primary visual routines, one for identifying the visual image near the fovea (object identification), and another for locating a stored prototype on the retina (object location). This design allows complex visual behaviors to be obtained by composing these two routines with different parameters. The iconic representations are comprised of high-dimensional feature vectors obtained from the responses of an ensemble of Gaussian derivative spatial filters at a number of orientations and scales. These representations are stored in two separate memories. One memory is indexed by image coordinates while the other is indexed by object coordinates. Object location matches a localized set of model features with image features at all possible retinal locations. Object identification matches a foveal set of image features with all possible model features. We present experimental results for a near real-time implementation of these routines on a pipeline image processor and suggest relatively simple strategies for tackling the problems of occlusions and scale variations. We also discuss two additional visual routines, one for top-down foveal targeting using log-polar sensors and another for looming detection, which are facilitated by the proposed architecture.

IJCAI Conference 1995 Conference Paper

Natural Basis Functions and Topographic Memory for Face Recognition

• Rajesh P N. Rao
• Dana H. Ballard

Recent work regarding the statistics of natural images has revealed that the dominant eigenvectors of arbitrary natural images closely approximate various oriented derivative-of- Gaussian functions; these functions have also been shown to provide the best fit to the receptive field profiles of cells in the primate striate cortex. We propose a scheme for expressioninvariant face recognition that employs a fixed set of these "natural" basis functions to generate multiscale iconic representations of human faces. Using a fixed set of basis functions obviates the need for recomputing eigenvectors (a step that was necessary in some previous approaches employing principal component analysis (PCA) for recognition) while at the same time retaining the redundancy-reducing properties of PCA. A face is represented by a set of iconic representations automatically extracted from an input image. The description thus obtained is stored in a topographically-organized sparse distributed memory that is based on a model of human long-term memory first proposed by Kanerva. We describe experimental results for an implementation of the method on a pipeline image processor that is capable of achieving near real-time recognition by exploiting the processor's frame-rate convolution capability for indexing purposes.

ICRA Conference 1995 Conference Paper

Remote Teleassistance

• Polly K. Pook
• Dana H. Ballard

We contrast the effects of communication latency on our human/robot control technique, called teleassistance, versus traditional teleoperation. In teleassistance, a human operator uses hand signs to guide an otherwise autonomous robot manipulator through a given task. Each sign signals a context switch and provides task-centered reference frames for the robot's autonomous servo-motor routines. The signs are natural, such as pointing to an object to indicate the desire to reach toward it as well as the axis along which to reach. The robot is a Utah/MIT hand mounted on a Puma 760 arm. For both teleassistance and teleoperation, the operator wears an EXOS hand master, a polhemus arm position sensor and a Virtual Research helmet that is coupled to binocular cameras mounted on a second Puma 760. The use of a video helmet allows for remote control of the robot and the simulation of communication lag-time. Experimental results suggest that teleassistance scales well with latency delays, unlike teleoperation.

IROS Conference 1994 Conference Paper

Deictic teleassistance

• Polly K. Pook
• Dana H. Ballard

We present a simple sign language for teleassistance inspired by the work of the Bernstein (1967) and by psychophysical evidence in hand-eye coordination. In our schema, a teleoperator uses hand signs to guide an otherwise autonomous robot manipulator through a given task. Each sign signals a context switch and provides a hand-centered reference frame for the robot's servomotor routines. The signs are natural, such as pointing to an object to indicate the desire to reach toward it as well as the axis along which to reach. These signs are called deictic from the Greek word for pointing to stress their indicative and relative nature. The task example is opening a door using a Utah/MIT hand mounted on a Puma 760 arm. The teleoperator wears an EXOS hand master and polhemus sensor. Three variations of nearest neighbor pattern classification are tested for online recognition of the sign language. The simplest, in which the operator signs each pose once before starting, is the best for this task. The dual-control strategy of teleassistance combines teleoperation and autonomous servo control to their advantage. The use of a symbolic sign language helps to alleviate many problems inherent to literal master/slave teleoperation. Conversely, the integration of global operator guidance and hand-centered coordinate frames permits the servo routines to position the robot in relative coordinates and interpret feedback within a constrained context, significantly simplifying the computation and reducing the need for detailed task models. >

ICML Conference 1994 Conference Paper

Hierarchical Self-Organization in Genetic programming

• Justinian Rosca
• Dana H. Ballard

AIJ Journal 1991 Journal Article

Animate vision

• Dana H. Ballard

Animate vision systems have gaze control mechanisms that can actively position the camera coordinate system in response to physical stimuli. Compared to passive systems, animate systems show that visual computation can be vastly less expensive when considered in the larger context of behavior. The most important visual behavior is the ability to control the direction of gaze. This allows the use of very low resolution imaging that has a high virtual resolution. Using such a system in a controlled way provides additional constraints that dramatically simplify the computations of early vision. Another important behavior is the way the environment “behaves”. Animate systems under real-time constraints can further reduce their computational burden by using environmental cues that are perspicuous in the local context. A third source of economy is introduced when behaviors are learned. Because errors are rarely fatal, systems using learning algorithms can amortize computational cost over extended periods. Further economies can be achieved when the learning system uses indexical reference, which is a form of dynamic variable binding. Animate vision is a natural way of implementing this dynamic binding.

IJCAI Conference 1989 Conference Paper

Reference Frames for Animate Vision

• Dana H. Ballard

AAAI Conference 1987 Conference Paper

Modular Learning in Neural Networks

• Dana H. Ballard

In the development of large-scale knowledge networks, much recent progress has been inspired by connections to neurobiology. An important component of any "neural" network is an accompanying learning algorithm. Such an algorithm, to be biologically plausible, must work for very large numbers of units. Studies of large-scale systems have so far been restricted to systems without internal units (units with no direct connections to the input or output). Internal units are crucial to such systems as they are the means by which a system can encode high-order regularities (or invariants) that are implicit in its inputs and outputs. Computer simulations of learning using internal units have been restricted to small-scale systems. This paper describes a way of coupling autoassociative learning modules into hierarchies that should greatly improve the performance of learning algorithms in large-scale systems. The idea has been tested experimentally with positive results.

AAAI Conference 1986 Conference Paper

Parallel Logical Inference and Energy Minimization

• Dana H. Ballard

The inference capabilities of humans suggest that they might be using algorithms with high degrees of parallelism. This paper develops a completely parallel connectionist inference mechanism. The mechanism handles obvious inferences, where each clause is only used once, but may be extendable to harder cases. The main contribution of this paper is to show formally that some inference can be reduced to an energy minimization problem in a way that is potentially useful.

ICRA Conference 1985 Conference Paper

Self-calibration in robot manipulators

• Amitabha Mukerjee
• Dana H. Ballard

The development of fast recursive methods for computing manipulator inverse dynamics has made possible open loop control strategies. However, for these strategies to work, an accurate plant model is required. Two key components of the plant are frictional terms and link/load inertias. This paper shows how these components may be computed using force and moment sensing.

IJCAI Conference 1985 Conference Paper

Transformational Form Perception in 3D: Constraints, Algorithms, Implementation

• Dana H. Ballard
• Hiromi Tanaka

AIJ Journal 1984 Journal Article

Parameter nets

• Dana H. Ballard

This paper describes the nucleus of a connectionist theory of low-level and intermediate-level vision. The theory explains segmentation in terms of massively parallel cooperative computation among intrinsic images and a set of feature networks at different levels of abstraction. Explaining how parts of an image are perceived as a meaningful whole or gestalt is a problem central to vision. A stepping stone towards a solution is recent work showing how to calculate images of physical parameters from intensity data. Such images are known as intrinsic images, and examples are images of velocity (optical flow), surface orientation, occluding contour, and disparity. Intrinsic images show great promise; they are distinctly easier to work with than the original intensity image, but they are not grouped into objects. A general way in which such groupings can be detected is to represent possible groupings as networks whose nodes signify explicit parameter values. In this case the relation between parts of an intrinsic image and the gestalt parameters can be specified by active, two-way connections between an intrinsic image network and a gestalt parameter network. The active connections will be many-to-one onto parameter value nodes that represent object features. The virtues of the methodology are that it can handle occlusion and noise and can specify intrinsic image boundaries. Furthermore, it can be made practical for high-dimensional feature spaces.

AAAI Conference 1984 Conference Paper

Task Frames in Robot Manipulation

• Dana H. Ballard

Most robotics computations refer to a single world- based frame of reference; however, several advantages accrue with the introduction of a second frame, termed a task frame. A task frame is a coordinate frame that can be attached to different objects that are to be manipulated. The task frame is related to the world-based coordinate frame by a simple geometric transformation. The virtues of such a frame are: (1) certain actions that are difficult to specify in the world frame are easily expressed in the task frame: (2) the task-frame to task-uorld transformation provides a formalism for describing physical actions; and (3) the task frame can be related to the world frame by proprioception.