Reid Simmons

IJCAI Conference 2025 Conference Paper

Sample-Efficient Behavior Cloning Using General Domain Knowledge

• Feiyu Zhu
• Jean Oh
• Reid Simmons

Behavior cloning has shown success in many sequential decision-making tasks by learning from expert demonstrations, yet they can be very sample inefficient and fail to generalize to unseen scenarios. One approach to these problems is to introduce general domain knowledge, such that the policy can focus on the essential features and may generalize to unseen states by applying that knowledge. Although this knowledge is easy to acquire from the experts, it is hard to be combined with learning from individual examples due to the lack of semantic structure in neural networks and the time-consuming nature of feature engineering. To enable learning from both general knowledge and specific demonstration trajectories, we use a large language model’s coding capability to instantiate a policy structure based on expert domain knowledge expressed in natural language and tune the parameters in the policy with demonstrations. We name this approach the Knowledge Informed Model (KIM) as the structure reflects the semantics of expert knowledge. In our experiments with lunar lander and car racing tasks, our approach learns to solve the tasks with as few as 5 demonstrations and is robust to action noise, outperforming the baseline model without domain knowledge. This indicates that with the help of large language models, we can incorporate domain knowledge into the structure of the policy, increasing sample efficiency for behavior cloning.

AAAI Conference 2024 Conference Paper

Bootstrapping Cognitive Agents with a Large Language Model

• Feiyu Zhu
• Reid Simmons

Large language models contain noisy general knowledge of the world, yet are hard to train or fine-tune. In contrast cognitive architectures have excellent interpretability and are flexible to update but require a lot of manual work to instantiate. In this work, we combine the best of both worlds: bootstrapping a cognitive-based model with the noisy knowledge encoded in large language models. Through an embodied agent doing kitchen tasks, we show that our proposed framework yields better efficiency compared to an agent entirely based on large language models. Our experiments also indicate that the cognitive agent bootstrapped using this framework can generalize to novel environments and be scaled to complex tasks.

IJCAI Conference 2021 Conference Paper

Interaction Considerations in Learning from Humans

• Pallavi Koppol
• Henny Admoni
• Reid Simmons

The ability to learn from large quantities of complex data has led to the development of intelligent agents such as self-driving cars and assistive devices. This data often comes from people via interactions such as labeling, providing rewards and punishments, and giving demonstrations or critiques. However, people's ability to provide high-quality data can be affected by human factors of an interaction, such as induced cognitive load and perceived usability. We show that these human factors differ significantly between interaction types. We first formalize interactions as a Markov Decision Process, and construct a taxonomy of these interactions to identify four archetypes: Showing, Categorizing, Sorting, and Evaluating. We then run a user study across two task domains. Our findings show that Evaluating interactions are more cognitively loading and less usable than the others, and Categorizing and Showing interactions are the least cognitively loading and most usable.

IJCAI Conference 2021 Conference Paper

Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning

• Yuchen Cui
• Pallavi Koppol
• Henny Admoni
• Scott Niekum
• Reid Simmons
• Aaron Steinfeld
• Tesca Fitzgerald

Human-in-the-loop Machine Learning (HIL-ML) is a widely adopted paradigm for instilling human knowledge in autonomous agents. Many design choices influence the efficiency and effectiveness of such interactive learning processes, particularly the interaction type through which the human teacher may provide feedback. While different interaction types (demonstrations, preferences, etc. ) have been proposed and evaluated in the HIL-ML literature, there has been little discussion of how these compare or how they should be selected to best address a particular learning problem. In this survey, we propose an organizing principle for HIL-ML that provides a way to analyze the effects of interaction types on human performance and training data. We also identify open problems in understanding the effects of interaction types.

IJCAI Conference 2009 Conference Paper

• Laura M. Hiatt
• Terry L. Zimmerman
• Stephen F. Smith
• Reid Simmons

In this paper, we describe an approach to scheduling under uncertainty that achieves scalability through a coupling of deterministic and probabilistic reasoning. Our specific focus is a class of oversubscribed scheduling problems where the goal is to maximize the reward earned by a team of agents in a distributed execution environment. There is uncertainty in both the duration and outcomes of executed activities. To ensure scalability, our solution approach takes as its starting point an initial deterministic schedule for the agents, computed using expected duration reasoning. This initial agent schedule is probabilistically analyzed to find likely points of failure, and then selectively strengthened based on this analysis. For each scheduled activity, the probability of failing and the impact that failure would have on the schedule’s overall reward are calculated and used to focus schedule strengthening actions. Such actions generally entail fundamental trade-offs; for example, modifications that increase the certainty that a high-reward activity succeeds may decrease the schedule slack available to accommodate uncertainty during execution. We describe a principled approach to handling these trade-offs based on the schedule’s “expected reward, ” using it as a metric to ensure that all schedule modifications are ultimately beneficial. Finally, we present experimental results obtained using a multi-agent simulation environment, which confirm that executing schedules strengthened in this way result in significantly higher rewards than are achieved by executing the corresponding initial schedules.

AAMAS Conference 2007 Conference Paper

Exploiting Factored Representations for Decentralized Execution in Multi-agent Teams

• Maayan Roth
• Reid Simmons
• Manuela Veloso

In many cooperative multi-agent domains, there exist some states in which the agents can act independently and others in which they need to coordinate with their teammates. In this paper, we explore how factored representations of state can be used to generate factored policies that can, with minimal communication, be executed distributedly by a multi-agent team. The factored policies indicate those portions of the state where no coordination is necessary, automatically alert the agents when they reach a state in which they do need to coordinate, and determine what the agents should communicate in order to achieve this coordination. We evaluate the success of our approach experimentally by comparing the amount of communication needed by a team executing a factored policy to a team that needs to communicate in every timestep.

AAMAS Conference 2005 Conference Paper

Reasoning About Joint Beliefs for Execution-Time Communication Decisions

• Roth Maayan
• Reid Simmons
• Manuela M. Veloso

AAMAS Conference 2003 Conference Paper

Task Allocation Using a Distributed Market-Based Planning Mechanism

• Dani Goldberg
• Vincent Cicirello
• M. Bernardine Dias
• Reid Simmons
• Stephen Smith
• Anthony Stentz

IJCAI Conference 2003 Conference Paper

Variable Resolution Particle Filter

• Vandi Verma
• Sebastian Thrun
• Reid Simmons

Particle filters are used extensively for tracking the state of non-linear dynamic systems. This paper presents a new particle filter that maintains samples in the state space at dynamically varying resolution for computational efficiency. Resolution within siatespace varies by region, depending on the belief that the true state lies within each region. Where belief is strong, resolution is fine. Where belief is low, resolution is coarse, abstracting multiple similar states together. The resolution of the statespace is dynamically updated as the belief changes. The proposed algorithm makes an explicit bias-variance tradeoff to select between maintaining samples in a biased generalization of a region of state space versus in a high variance specialization at fine resolution. Samples are maintained at a coarser resolution when the bias introduced by the generalization to a coarse resolution is outweighed by the gain in terms of reduction in variance, and at a finer resolution when it is not. Maintaining samples in abstraction prevents potential hypotheses from being eliminated prematurely for lack of a sufficient number of particles. Empirical results show that our variable resolution particle filter requires significantly lower computation for performance comparable to a classical particle filter.

AAAI Conference 1996 Conference Paper

Amelia

• Reid Simmons
• Greg Armstrong
• Karen Haigh
• Shyjan Mahamud

Amelia was built by Real World Interface (RWI) using Xavier-a mobile robot platform developed at CMU on a B24 base from RWI-as a prototype. Amelia has substantial engineering improvements over Xavier. Amelia is built on a B21 base. It has a top speed of 32 inches per second, while improved integral dead-reckoning insures extremely accurate drive and position controls.

IJCAI Conference 1995 Conference Paper

Probabilistic Robot Navigation in Partially Observable Environments

• Reid Simmons
• Sven Koenig

IJCAI Conference 1987 Conference Paper

Generate, Test and Debug: Combining Associational Rules and Causal Models

• Reid Simmons
• Randall Davis

AAAI Conference 1986 Conference Paper

Commonsense Arithmetic Reasoning

• Reid Simmons

“Arithmetic reasoning” can range in complexity from simple integer arithmetic to powerful symbolic algebraic reasoning of the sort done by MACSYMA. We describe an arithmetic reasoning system of intermediate complexity called the Quantity Lattice. In a computationally efficient manner the Quantity Lattice integrates qualitative and quantitative reasoning, and combines inequality reasoning with reasoning about simple arithmetic expressions, such as addition or multiplication. The system has proven useful in doing simulation and analysis in several domains, including geology and semiconductor fabrication, by supporting useful forms of reasoning about time and the changes that hap pen when processes occur.