Lawrence Birnbaum

TMLR Journal 2026 Journal Article

ActionEQA: Action Interface for Embodied Question Answering

• Tianwei Bao
• Qineng Wang
• Kangrui Wang
• Mingkai Deng
• Guangyi Liu
• Jiayuan Mao
• Lawrence Birnbaum
• Zhiting Hu

While Vision-Language Models (VLMs) are increasingly integral to embodied intelligence, a significant action understanding bottleneck persists in translating high-level semantic instructions into precise low-level physical actions. However, current benchmarks for embodied agents primarily focus on high-level perception and planning, failing to capture the depth and nature of this semantic-to-physical gap. To address this, we introduce ActionEQA, the first Embodied Question Answering (EQA) benchmark designed to methodically evaluate the ability of VLMs to bridge this critical yet underexplored semantic-physical divide. Grounded in real-world robotics data, ActionEQA thoroughly analyzes VLMs’ grasp of the action interface using a dual-tier design: (1) a Three-Tiered Action Hierarchy for pinpointing the depth at which VLMs' action reasoning collapses. (2) Bidirectional Reasoning Tasks for testing whether VLMs struggle more to predict action outcomes or infer the actions that led to them. Our key findings reveal: (1) The primary bottleneck in action understanding occurs at the mid-level, arising from the challenge of grounding compositional language in 3D physical geometry. (2) VLMs are more adept at inferring past actions than predicting their future outcomes. (3) Richer visual inputs require greater spatial reasoning from VLMs to map actions to physical geometry. (4) Within the action hierarchy, model failures shift from predominantly perceptual errors at the high level to flawed geometric and physical reasoning at the low level.

AAAI Conference 2007 Conference Paper

Integrating Natural Language, Knowledge Representation and Reasoning, and Analogical Processing to Learn by Reading

• Kenneth D. Forbus
• Lawrence Birnbaum
• Abhishek Sharma

Learning by reading requires integrating several strands of AI research. We describe a prototype system, Learning Reader, which combines natural language processing, a large-scale knowledge base, and analogical processing to learn by reading simplified language texts. We outline the architecture of Learning Reader and some of system-level results, then explain how these results arise from the components. Specifically, we describe the design, implementation, and performance characteristics of a natural language understanding model (DMAP) that is tightly coupled to a knowledge base three orders of magnitude larger than previous attempts. We show that knowing the kinds of questions being asked and what might be learned can help provide more relevant, efficient reasoning. Finally, we show that analogical processing provides a means of generating useful new questions and conjectures when the system ruminates off-line about what it has read.

AIJ Journal 1993 Journal Article

Retrospective on “The organization of expert systems, a tutorial”

• Mark Stefik
• Jan S. Aikins
• Robert Balzer
• John Benoit
• Lawrence Birnbaum
• Frederick Hayes-Roth
• Earl D. Sacerdoti

IJCAI Conference 1991 Conference Paper

Plan Debugging in an Intentional System

• Gregg Collins
• Lawrence Birnbaum
• Bruce Krulwich
• Michael

We have developed a model-based approach to learning from plan failures in which an agent uses a model of itself to determine where in its planning or execution the cause of a failure lies. We believe that such an approach constitutes the most promising basis for developing learning models that are capable of deciding for themselves what needs to be learned from a given experience. In addition, such methods appear capable of learning about planning at a very abstract level, and thus of supporting the transfer of knowledge from experience in one domain or task to other domains or tasks.

AIJ Journal 1991 Journal Article

Rigor mortis: A response to Nilsson's “Logic and artificial intelligence”

• Lawrence Birnbaum

Logicism has contributed greatly to progress in AI by emphasizing the central role of mental content and representational vocabulary in intelligent systems. Unfortunately, the logicists' dream of a completely use-independent characterization of knowledge has drawn their attention away from these fundamental AI problems, leading instead to a concentration on purely formalistic issues in deductive inference and model-theoretic “semantics”. In addition, their failure to resist the lure of formalistic modes of expression has unnecessarily curtailed the prospects for intellectual interaction with other AI researchers.

AAAI Conference 1990 Conference Paper

Model-Based Diagnosis of Planning Failures

• Lawrence Birnbaum
• Michael Freed

We propose that a planner should be provided with an explicit model of its own planning mechanism, and show that linking a planner’ s expectations about the performance of its plans to such a model, by means of explicit justification structures, enables the planner to determine which aspects of its planning are responsible for observed performance failures. We have implemented the ideas presented in this paper in a computer model. Applied to the game of chess, the model is capable of diagnosing planning failures due to incomplete knowledge of the rules, improper or overly optimistic focus of attention, faulty projection, and insufficient lead time for warning about threats, and is therefore able to learn such concepts as discovered attack and the fork.

IJCAI Conference 1989 Conference Paper

An Adaptive Model of Decision-Making in Planning

• Gregg Collins
• Lawrence Birnbaum
• Bruce Krulwich

Learning how to make decisions in a domain is a critical aspect of intelligent planning behavior. The ability of a planner to adapt its decision-making to a domain depends in part upon its ability to optimize the tradeoff between the sophistication of its decision procedures and their cost. Since it is difficult to optimize this tradeoff on a priori grounds alone, we propose that a planner start with a relatively simple set of decision procedures, and add complexity in response to experience gained in the application of its decision-making to real-world problems. Our model of this adaptation process is based on the explanation of failures, in that it is the analysis of bad decisions that drives the improvement of the decision procedures. We have developed a test-bed system for the implementation of planning models employing such an approach, and have demonstrated the ability of such a model to improve its procedure for projecting the effects of its moves in chess.

IJCAI Conference 1985 Conference Paper

A Short Note on Opportunistic Planning and Memory in Arguments

• Lawrence Birnbaum

IJCAI Conference 1985 Conference Paper

Lexical Ambiguity as a Touchstone for Theories of Language Analysis

• Lawrence Birnbaum

This paper assesses several broad approaches to language analysis with respect to the problem of lexical ambiguity. The impact of the problem on both syntactic and semantic analysis is discussed, and several common methods for disambiguation, including the use of selectional restrictions and scriptal lexicons, are analyzed. Their shortcomings illustrate the need for complex inference to resolve ambiguity, which forms one of the key functional arguments in favor of integrating language analysis with memory and inference. However, it has proven surprisingly difficult to realize such an integrated approach in practice: An assessment of lexical disambiguation within some recent models which attempt to do so reveals that they rely largely on the traditional techniques of selectional restrictions and scriptal lexicons, with all their drawbacks. The difficulty is shown to stem primarily from the theories of memory and inferential processing utilized. The implications for recent approaches to language analysis based on connectionist mechanisms are explored. Finally, the requirements imposed by lexical disambiguation on theories of memory and inferential processing are discussed.

AAAI Conference 1982 Conference Paper

Argument Molecules: A Functional Representation of Argument Structure

• Lawrence Birnbaum

Understanding an utterance in an argument crucially requires determining the evidential relations it bears to prior and subsequent propositions in the argument (Birnbaum et al., 1980; Cohen, 1981). The memory representation of an argument should, accordingly, indicate which propositions a given proposition counts as evidence for (a support relation) or against (an attack relation), and which propositions support or attack it in turn. The representation of an argument can thus be viewed as a network of propositions connected by support or attack relations (an argument graph). Although this sort of representation can be motivated simply by the need to represent the content of an argument, it seems natural to ask whether such argument graphs might further possess any useful structural properties, abstracted from the specific propositions they relate.

AIJ Journal 1982 Journal Article

The organization of expert systems, a tutorial

• Mark Stefik
• Jan Aikins
• Robert Balzer
• John Benoit
• Lawrence Birnbaum
• Frederick Hayes-Roth
• Earl Sacerdoti

This is a tutorial about the organization of expert problem-solving programs. We begin with a restricted class of problems that admits a very simple organization. To make this organization feasible it is required that the input data be static and reliable and that the solution space be small enough to search exhaustively. These assumptions are then relaxed, one at a time, in case study of ten more sophisticated organizational prescriptions. The first cases give techniques for dealing with unreliable data and time-varying data. Other cases show techniques for creating and reasoning with abstract solution spaces and using multiple lines of reasoning. The prescriptions are compared for their coverage and illustrated by examples from recent expert systems.

IJCAI Conference 1981 Conference Paper

Using Language and Context in the Analysis of Text

• Opportunistic Processing in Arguments Rod McGuire
• Lawrence Birnbaum

AAAI Conference 1980 Conference Paper

Towards an AI Model of Argumentation

• Lawrence Birnbaum

This paper describes a process model of human argumentation, and provides examples of its operation as implemented in a computer program. Our main concerns include such issues as the rules and structures underlying argumentation, how these relate to conversational rules, how reasoning is used in arguments, and how arguing and reasoning interact. implemented, and using them the program is capable of participating in the following argument fragment, concerning the question of who was responsible for the 1967 Arab-Israeli war. The program can assume either the Israeli or the Arab point of view. [ll Arab: Who started the 1967 War? [21 Israeli: The Arabs did, by blockading the Straits of Tiran.