Author name cluster

Dana Ballard

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

1 author row

NeurIPS Conference 2021 Conference Paper

Machine versus Human Attention in Deep Reinforcement Learning Tasks

Suna (Sihang) Guo
Ruohan Zhang
Bo Liu
Yifeng Zhu
Dana Ballard
Mary Hayhoe
Peter Stone

Deep reinforcement learning (RL) algorithms are powerful tools for solving visuomotor decision tasks. However, the trained models are often difficult to interpret, because they are represented as end-to-end deep neural networks. In this paper, we shed light on the inner workings of such trained models by analyzing the pixels that they attend to during task execution, and comparing them with the pixels attended to by humans executing the same tasks. To this end, we investigate the following two questions that, to the best of our knowledge, have not been previously studied. 1) How similar are the visual representations learned by RL agents and humans when performing the same task? and, 2) How do similarities and differences in these learned representations explain RL agents' performance on these tasks? Specifically, we compare the saliency maps of RL agents against visual attention models of human experts when learning to play Atari games. Further, we analyze how hyperparameters of the deep RL algorithm affect the learned representations and saliency maps of the trained agents. The insights provided have the potential to inform novel algorithms for closing the performance gap between human experts and RL agents.

PDF Details

AAAI Conference 2020 Conference Paper

Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset

Ruohan Zhang
Calen Walshe
Zhuode Liu
Lin Guan
Karl Muller
Jake Whritner
Luxin Zhang
Mary Hayhoe

Large-scale public datasets have been shown to beneﬁt research in multiple areas of modern artiﬁcial intelligence. For decision-making research that requires human data, highquality datasets serve as important benchmarks to facilitate the development of new methods by providing a common reproducible standard. Many human decision-making tasks require visual attention to obtain high levels of performance. Therefore, measuring eye movements can provide a rich source of information about the strategies that humans use to solve decision-making tasks. Here, we provide a largescale, high-quality dataset of human actions with simultaneously recorded eye movements while humans play Atari video games. The dataset consists of 117 hours of gameplay data from a diverse set of 20 games, with 8 million action demonstrations and 328 million gaze samples. We introduce a novel form of gameplay, in which the human plays in a semiframe-by-frame manner. This leads to near-optimal game decisions and game scores that are comparable or better than known human records. We demonstrate the usefulness of the dataset through two simple applications: predicting human gaze and imitating human demonstrated actions. The quality of the data leads to promising results in both tasks. Moreover, using a learned human gaze model to inform imitation learning leads to an 115% increase in game performance. We interpret these results as highlighting the importance of incorporating human visual attention in models of decision making and demonstrating the value of the current dataset to the research community. We hope that the scale and quality of this dataset can provide more opportunities to researchers in the areas of visual attention, imitation learning, and reinforcement learning.

PDF Details

IJCAI Conference 2020 Conference Paper

Human Gaze Assisted Artificial Intelligence: A Review

Ruohan Zhang
Akanksha Saran
Bo Liu
Yifeng Zhu
Sihang Guo
Scott Niekum
Dana Ballard
Mary Hayhoe

Human gaze reveals a wealth of information about internal cognitive state. Thus, gaze-related research has significantly increased in computer vision, natural language processing, decision learning, and robotics in recent years. We provide a high-level overview of the research efforts in these fields, including collecting human gaze data sets, modeling gaze behaviors, and utilizing gaze information in various applications, with the goal of enhancing communication between these research areas. We discuss future challenges and potential applications that work towards a common goal of human-centered artificial intelligence.

PDF Details DOI

AAAI Conference 2018 Short Paper

Learning Attention Model From Human for Visuomotor Tasks

Luxin Zhang
Ruohan Zhang
Zhuode Liu
Mary Hayhoe
Dana Ballard

A wealth of information regarding intelligent decision making is conveyed by human gaze and visual attention, hence, modeling and exploiting such information might be a promising way to strengthen algorithms like deep reinforcement learning. We collect high-quality human action and gaze data while playing Atari games. Using these data, we train a deep neural network that can predict human gaze positions and visual attention with high accuracy.

PDF Details

AAAI Conference 2015 Conference Paper

Global Policy Construction in Modular Reinforcement Learning

Ruohan Zhang
Zhao Song
Dana Ballard

We propose a modular reinforcement learning algorithm which decomposes a Markov decision process into independent modules. Each module is trained using Sarsa(λ). We introduce three algorithms for forming global policy from modules policies, and demonstrate our results using a 2D grid world.

PDF Details

RLDM Conference 2015 Conference Abstract

Modular Inverse Reinforcement Learning on Human Motion

Shun Zhang
Matthew Tong
Mary Hayhoe
Dana Ballard

Reinforcement learning has been seen as a useful model for understanding human behavior because of the importance of the neural reward circuitry. However, because of the difficulty of scaling up RL models to large state spaces, it has been hard to apply these models to complex human behaviors. One potential simplification, consistent with observations of natural behavior, is that complex tasks can be broken down into independent sub-tasks, or modules. In this paper, we use observed human behavior while walking along a path to estimate the intrinsic reward values associated with different modular sub-tasks. To do this we use a simplified version of Inverse Reinforcement Learning to calculate the reward associated with path following, obstacle avoidance, and target collection of humans acting in an immersive virtual environment. Using the estimated values, a modular RL model can generate realistic behavior consistent with human action choices. This provides a way of understanding momentary sensorimotor decisions made in complex natural environments.

PDF Details

AAAI Conference 2014 Conference Paper

Efficient Codes for Inverse Dynamics During Walking

Leif Johnson
Dana Ballard

Efficient codes have been used effectively in both computer science and neuroscience to better understand the information processing in visual and auditory encoding and discrimination tasks. In this paper, we explore the use of efficient codes for representing information relevant to human movements during locomotion. Specifically, we apply motion capture data to a physical model of the human skeleton to compute joint angles (inverse kinematics) and joint torques (inverse dynamics); then, by treating the resulting paired dataset as a supervised regression problem, we investigate the effect of sparsity in mapping from angles to torques. The results of our investigation suggest that sparse codes can indeed represent salient features of both the kinematic and dynamic views of human locomotion movements. However, sparsity appears to be only one parameter in building a model of inverse dynamics; we also show that the ”encoding” process benefits significantly by integrating with the ”regression” process for this task. In addition, we show that, for this task, simple coding and decoding methods are not sufficient to model the extremely complex inverse dynamics mapping. Finally, we use our results to argue that representations of movement are critical to modeling and understanding these movements.

PDF Details

NeurIPS Conference 2003 Conference Paper

Eye Movements for Reward Maximization

Nathan Sprague
Dana Ballard

University of Rochester Rochester, NY 14627 dana@cs. rochester. edu Recent eye tracking studies in natural tasks suggest that there is a tight link between eye movements and goal directed motor actions. However, most existing models of human eye movements provide a bottom up ac- count that relates visual attention to attributes of the visual scene. The purpose of this paper is to introduce a new model of human eye move- ments that directly ties eye movements to the ongoing demands of be- havior. The basic idea is that eye movements serve to reduce uncertainty about environmental variables that are task relevant. A value is assigned to an eye movement by estimating the expected cost of the uncertainty that will result if the movement is not made. If there are several candidate eye movements, the one with the highest expected value is chosen. The model is illustrated using a humanoid graphic ﬁgure that navigates on a sidewalk in a virtual urban environment. Simulations show our protocol is superior to a simple round robin scheduling mechanism.

PDF Details

IJCAI Conference 2003 Conference Paper

Multiple-Goal Reinforcement Learning with Modular Sarsa(O)

Nathan Sprague
Dana Ballard

We present a new algorithm, GM-Sarsa(O), for finding approximate solutions to multiple-goal reinforcement learning problems that are modeled as composite Markov decision processes. According to our formulation different sub-goals are modeled as MDPs that are coupled by the requirement that they share actions. Existing reinforcement learning algorithms address similar problem formulations by first finding optimal policies for the component MDPs, and then merging these into a policy for the composite task. The problem with such methods is that policies that are optimized separately may or may not perform well when they are merged into a composite solution. Instead of searching for optimal policies for the component MDPs in isolation, our approach finds good policies in the context of the composite task. keywords: reinforcement learning

PDF

NeurIPS Conference 1999 Conference Paper

Recognizing Evoked Potentials in a Virtual Environment

Jessica Bayliss
Dana Ballard

Virtual reality (VR) provides immersive and controllable experimen(cid: 173) tal environments. It expands the bounds of possible evoked potential (EP) experiments by providing complex, dynamic environments in or(cid: 173) der to study cognition without sacrificing environmental control. VR also serves as a safe dynamic testbed for brain-computer. interface (BCl) research. However, there has been some concern about detecting EP sig(cid: 173) nals in a complex VR environment. This paper shows that EPs exist at red, green, and yellow stop lights in a virtual driving environment. Ex(cid: 173) perimental results show the existence of the P3 EP at "go" and "stop" lights and the contingent negative variation (CNY) EP at "slow down" lights. In order to test the feasibility of on-line recognition in VR, we looked at recognizing the P3 EP at red stop tights and the absence of this signal at yellow slow down lights. Recognition results show that the P3 may successfully be used to control the brakes of a VR car at stop lights.

PDF Details

NeurIPS Conference 1995 Conference Paper

Modeling Saccadic Targeting in Visual Search

Rajesh Rao
Gregory Zelinsky
Mary Hayhoe
Dana Ballard

Visual cognition depends criticalIy on the ability to make rapid eye movements known as saccades that orient the fovea over targets of interest in a visual scene. Saccades are known to be ballistic: the pattern of muscle activation for foveating a prespecified target location is computed prior to the movement and visual feedback is precluded. Despite these distinctive properties, there has been no general model of the saccadic targeting strategy employed by the human visual system during visual search in natural scenes. This paper proposes a model for saccadic targeting that uses iconic scene representations derived from oriented spatial filters at multiple scales. Visual search proceeds in a coarse-to-fine fashion with the largest scale filter responses being compared first. The model was empirically tested by comparing its perfonnance with actual eye movement data from human subjects in a natural visual search task; preliminary results indicate substantial agreement between eye movements predicted by the model and those recorded from human subjects.

PDF Details

NeurIPS Conference 1994 Conference Paper

Learning Saccadic Eye Movements Using Multiscale Spatial Filters

Rajesh Rao
Dana Ballard

We describe a framework for learning saccadic eye movements using a photometric representation of target points in natural scenes. The rep(cid: 173) resentation takes the form of a high-dimensional vector comprised of the responses of spatial filters at different orientations and scales. We first demonstrate the use of this response vector in the task of locating pre(cid: 173) viously foveated points in a scene and subsequently use this property in a multisaccade strategy to derive an adaptive motor map for delivering accurate saccades.

PDF Details

NeurIPS Conference 1992 Conference Paper

A Note on Learning Vector Quantization

Virginia de
Dana Ballard

Vector Quantization is useful for data compression. Competitive Learn(cid: 173) ing which minimizes reconstruction error is an appropriate algorithm for vector quantization of unlabelled data. Vector quantization of labelled data for classification has a different objective, to minimize the number of misclassifications, and a different algorithm is appropriate. We show that a variant of Kohonen's LVQ2. 1 algorithm can be seen as a multi(cid: 173) class extension of an algorithm which in a restricted 2 class case can be proven to converge to the Bayes optimal classification boundary. We compare the performance of the LVQ2. 1 algorithm to that of a modified version having a decreasing window and normalized step size, on a ten class vowel classification problem.

PDF Details

NeurIPS Conference 1988 Conference Paper

Fixed Point Analysis for Recurrent Networks

Patrice Simard
Mary Ottaway
Dana Ballard

This paper provides a systematic analysis of the recurrent backpropaga(cid: 173) tion (RBP) algorithm, introducing a number of new results. The main limitation of the RBP algorithm is that it assumes the convergence of the network to a stable fixed point in order to backpropagate the error signals. We show by experiment and eigenvalue analysis that this condi(cid: 173) tion can be violated and that chaotic behavior can be avoided. Next we examine the advantages of RBP over the standard backpropagation al(cid: 173) gorithm. RBP is shown to build stable fixed points corresponding to the input patterns. This makes it an appropriate tool for content address(cid: 173) able memories, one-to-many function learning, and inverse problems.

PDF Details