James Allen

AAAI Conference 2020 Short Paper

Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract)

• Ritwik Bose
• Siddharth Vashishtha
• James Allen

A Semantic Parser generates a logical form graph from an utterance where the edges are semantic roles and nodes are word senses in an ontology that supports reasoning. The generated representation attempts to capture the full meaning of the utterance. While the process of parsing works to resolve lexical ambiguity, a number of errors in the logical forms arise from incorrectly assigned word sense determinations. This is especially true in logical and rule-based semantic parsers. Although the performance of statistical word sense disambiguation methods is superior to the word sense output of semantic parser, these systems do not produce the rich role structure or a detailed semantic representation of the sentence content. In this work, we use decisions from a statistical WSD system to inform a logical semantic parser and greatly improve semantic type assignments in the resulting logical forms.

AAAI Conference 2018 Conference Paper

Effective Broad-Coverage Deep Parsing

• James Allen
• Omid Bahkshandeh
• William de Beaumont
• Lucian Galescu
• Choh Man Teng

Current semantic parsers either compute shallow representations over a wide range of input, or deeper representations in very limited domains. We describe a system that provides broad-coverage, deep semantic parsing designed to work in any domain using a core domain-general lexicon, ontology and grammar. This paper discusses how this core system can be customized for a particularly challenging domain, namely reading research papers in biology. We evaluate these customizations with some ablation experiments.

AAAI Conference 2017 System Paper

Natural Language Dialogue for Building and Learning Models and Structures

• Ian Perera
• James Allen
• Lucian Galescu
• Choh Man Teng
• Mark Burstein
• Scott Friedman
• David McDonald
• Jeffrey Rye

We demonstrate an integrated system for building and learning models and structures in both a real and virtual environment. The system combines natural language understanding, planning, and methods for composition of basic concepts into more complicated concepts. The user and the system interact via natural language to jointly plan and execute tasks involving building structures, with clarifications and demonstrations to teach the system along the way. We use the same architecture for building and simulating models of biology, demonstrating the general-purpose nature of the system where domain-specific knowledge is concentrated in sub-modules with the basic interaction remaining domain-independent. These capabilities are supported by our work on semantic parsing, which generates knowledge structures to be grounded in a physical representation, and composed with existing knowledge to create a dynamic plan for completing goals. Prior work on learning from natural language demonstrations enables learning of models from very few demonstrations, and features are extracted from definitions in natural language. We believe this architecture for interaction opens up a wide possibility of human-computer interaction and knowledge transfer through natural language.

AAAI Conference 2013 Conference Paper

Integrating Programming by Example and Natural Language Programming

• Mehdi Manshadi
• Daniel Gildea
• James Allen

We motivate the integration of programming by example and natural language programming by developing a system for specifying programs for simple text editing operations based on regular expressions. The programs are described with unconstrained natural language instructions, and providing one or more examples of input/output. We show that natural language allows the system to deduce the correct program much more often and much faster than is possible with the input/output example(s) alone, showing that natural language programming and programming by example can be combined in a way that overcomes the ambiguities that both methods suffer from individually and, at the same time, provides a more natural interface to the user.

AAAI Conference 2013 Conference Paper

SALL-E: Situated Agent for Language Learning

• Ian Perera
• James Allen

We describe ongoing research towards building a cognitively plausible system for near one-shot learning of the meanings of attribute words and object names, by grounding them in a sensory model. The system learns incrementally from human demonstrations recorded with the Microsoft Kinect, in which the demonstrator can use unrestricted natural language descriptions. We achieve near-one shot learning of simple objects and attributes by focusing solely on examples where the learning agent is confident, ignoring the rest of the data. We evaluate the system’s learning ability by having it generate descriptions of presented objects, including objects it has never seen before, and comparing the system response against collected human descriptions of the same objects. We propose that our method of retrieving object examples with a k-nearest neighbor classifier using Mahalanobis distance corresponds to a cognitively plausible representation of objects. Our initial results show promise for achieving rapid, near one-shot, incremental learning of word meanings.

AAAI Conference 2012 Conference Paper

Learning Names for RFID-Tagged Objects in Activity Videos

• Ian Perera
• James Allen

We describe a method for determining the names of RFID-tagged objects in activity videos using descriptions which have been parsed to provide anaphoric reference resolution and ontological categorization.

AAAI Conference 2012 Conference Paper

Real-Time Collaborative Planning with the Crowd

• Walter Lasecki
• Jeffrey Bigham
• James Allen
• George Ferguson

Planning is vital to a wide range of domains, including robotics, military strategy, logistics, itinerary generation and more, that both humans and computers find difficult. Collaborative planning holds the promise of greatly improving performance on these tasks by leveraging the strengths of both humans and automated planners. However, this requires formalizing the problem domain and input, which must be done by hand, a priori, restricting its use in general real-world domains. We propose using a real-time crowd of workers to simultaneously solve the planning problem, formalize the domain, and train an automated system. As plans are developed, the system is able to learn the domain, and contribute larger segments of work.

AAAI Conference 2007 Conference Paper

PLOW: A Collaborative Task Learning Agent

• James Allen
• George Ferguson
• Hyuckchul Jung

To be effective, an agent that collaborates with humans needs to be able to learn new tasks from humans they work with. This paper describes a system that learns executable task models from a single collaborative learning session consisting of demonstration, explanation and dialogue. To accomplish this, the system integrates a range of AI technologies: deep natural language understanding, knowledge representation and reasoning, dialogue systems, planning/agent-based systems and machine learning. A formal evaluation shows the approach has great promise.

IJCAI Conference 2003 Conference Paper

Corpus-based, Statistical Goal Recognition

• Nate Blaylock
• James Allen

Goal recognition for dialogue systems needs to be fast, make early predictions, and be portable. We present initial work which shows that using statistical, corpus-based methods to build goal recognizers may be a viable way to meet those needs. Our goal recognizer is trained on data from apian corpus and then used to determine the agent's most likely goal based on that data. The algorithm is linear in the number of goals, and performs very well in terms of accuracy and early prediction. In addition, it is more easily portable to new domains as does not require a hand-crafted plan library.

AAMAS Conference 2002 Conference Paper

A Problem Solving Model for Collaborative Agents

• James Allen
• Nate Blaylock
• George Ferguson

AAAI Conference 1999 Conference Paper

Simulation-Based Inference for Plan Monitoring

• Neal Lesh
• MERL - A Mitsubishi Electric Research Laboratory
• James Allen
• University of Rochester

Thedynamicexecution of plans in uncertain domains requires the ability to infer likely current andfuture worldstates from past observations. Wecast this task as inference on DynamicBelief Networks(DBNs)but the resulting networksare difficult to solve with exact methods. Weinvestigate and extend simulation algorithms for approximateinference on Bayesiannetworks and propose a newalgorithm, called Rewind/Replay, for generating a set of simulations weightedby their likelihood givenpast observations. Wevalidate our algorithm on a DBN containing thousands of variables, whichmodelsthe spread of wildfire.