Author name cluster

Subhro Roy

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

NeurIPS Conference 2023 Conference Paper

BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing

Subhro Roy
Samuel Thomson
Tongfei Chen
Richard Shin
Adam Pauls
Jason Eisner
Benjamin Van Durme

Recent work has shown that generation from a prompted or fine-tuned language model can perform well at semantic parsing when the output is constrained to be a valid semantic representation. We introduce BenchCLAMP, a Benchmark to evaluate Constrained LAnguage Model Parsing, that includes context-free grammars for seven semantic parsing datasets and two syntactic parsing datasets with varied output meaning representations, as well as a constrained decoding interface to generate only valid outputs covered by these grammars. We provide low, medium, and high resource splits for each dataset, allowing accurate comparison of various language models under different data regimes. Our benchmark supports evaluation of language models using prompt-based learning as well as fine-tuning. We benchmark seven language models, including two GPT-3 variants available only through an API. Our experiments show that encoder-decoder pretrained language models can achieve similar performance or even surpass state-of-the-art methods for both syntactic and semantic parsing when the model output is constrained to be valid.

PDF Details

RLDM Conference 2019 Conference Abstract

Joint Goal and Constraint Inference using Bayesian Nonparametric Inverse Reinforcement Learning

Daehyung Park
Michael Noseworthy
Rohan Paul
Subhro Roy
Nicholas Roy

Inverse Reinforcement Learning (IRL) aims to recover an unknown reward function from expert demonstrations of a task. Often, the reward function fails to capture a complex behavior (e. g. , a condi- tion or a constraint) due to the simple structure of the global reward function. We introduce an algorithm, Constraint-based Bayesian Non-Parametric Inverse Reinforcement Learning (CBN-IRL), that instead repre- sents a task as a sequence of subtasks, each consisting of a goal and set of constraints, by partitioning a single demonstration into individual trajectory segments. CBN-IRL is able to find locally consistent constraints and adapt the number of subtasks according to the complexity of the demonstration using a computationally efficient inference process. We evaluate the proposed framework on two-dimensional simulation environ- ments. The results show our framework outperforms state-of-the-art IRL on a complex demonstration. We also show we can adapt the learned subgoals and constraints to randomized test environments given a single demonstration.

PDF Details

AAAI Conference 2017 Conference Paper

Unit Dependency Graph and Its Application to Arithmetic Word Problem Solving

Subhro Roy
Dan Roth

We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art. Our model has the additional advantage of being very interpretable, since it allows visualization of its predictions broken up by abstract features such as information content, salience and novelty. Another novel contribution of our work is abstractive training of our extractive model that can train on human generated reference summaries alone, eliminating the need for sentence-level extractive labels.

PDF Details