Author name cluster

Maxwell I. Nye

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

ICML Conference 2021 Conference Paper

A large-scale benchmark for few-shot program induction and synthesis

Ferran Alet
Javier Lopez-Contreras
James Koppel
Maxwell I. Nye
Armando Solar-Lezama
Tomás Lozano-Pérez
Leslie Pack Kaelbling
Joshua B. Tenenbaum

A landmark challenge for AI is to learn flexible, powerful representations from small numbers of examples. On an important class of tasks, hypotheses in the form of programs provide extreme generalization capabilities from surprisingly few examples. However, whereas large natural few-shot learning image benchmarks have spurred progress in meta-learning for deep networks, there is no comparably big, natural program-synthesis dataset that can play a similar role. This is because, whereas images are relatively easy to label from internet meta-data or annotated by non-experts, generating meaningful input-output examples for program induction has proven hard to scale. In this work, we propose a new way of leveraging unit tests and natural inputs for small programs as meaningful input-output examples for each sub-program of the overall program. This allows us to create a large-scale naturalistic few-shot program-induction benchmark and propose new challenges in this domain. The evaluation of multiple program induction and synthesis algorithms points to shortcomings of current methods and suggests multiple avenues for future work.

Details

ICLR Conference 2021 Conference Paper

Representing Partial Programs with Blended Abstract Semantics

Maxwell I. Nye
Yewen Pu
Matthew Bowers
Jacob Andreas
Joshua B. Tenenbaum
Armando Solar-Lezama

Synthesizing programs from examples requires searching over a vast, combinatorial space of possible programs. In this search process, a key challenge is representing the behavior of a partially written program before it can be executed, to judge if it is on the right track and predict where to search next. We introduce a general technique for representing partially written programs in a program synthesis engine. We take inspiration from the technique of abstract interpretation, in which an approximate execution model is used to determine if an unfinished program will eventually satisfy a goal specification. Here we learn an approximate execution model implemented as a modular neural network. By constructing compositional program representations that implicitly encode the interpretation semantics of the underlying programming language, we can represent partial programs using a flexible combination of concrete execution state and learned neural representations, using the learned approximate semantics when concrete semantics are not known (in unfinished parts of the program). We show that these hybrid neuro-symbolic representations enable execution-guided synthesizers to use more powerful language constructs, such as loops and higher-order functions, and can be used to synthesize programs more accurately for a given search budget than pure neural approaches in several domains.

Details

ICML Conference 2019 Conference Paper

Learning to Infer Program Sketches

Maxwell I. Nye
Luke B. Hewitt
Joshua B. Tenenbaum
Armando Solar-Lezama

Our goal is to build systems which write code automatically from the kinds of specifications humans can most easily provide, such as examples and natural language instruction. The key idea of this work is that a flexible combination of pattern recognition and explicit reasoning can be used to solve these complex programming problems. We propose a method for dynamically integrating these types of information. Our novel intermediate representation and training algorithm allow a program synthesis system to learn, without direct supervision, when to rely on pattern recognition and when to perform symbolic search. Our model matches the memorization and generalization performance of neural synthesis and symbolic search, respectively, and achieves state-of-the-art performance on a dataset of simple English description-to-code programming problems.

Details

UAI Conference 2018 Conference Paper

The Variational Homoencoder: Learning to learn high capacity generative models from few examples

Luke B. Hewitt
Maxwell I. Nye
Andreea Gane
Tommi S. Jaakkola
Joshua B. Tenenbaum

reused across many tasks. Recent work has approached one- and few-shot learning from all of these perspectives. Hierarchical Bayesian methods can unify many related tasks (e. g. k-shot classification, conditional and unconditional generation) as inference within a single generative model. However, when this generative model is expressed as a powerful neural network such as a PixelCNN, we show that existing learning techniques typically fail to effectively use latent variables. To address this, we develop a modification of the Variational Autoencoder in which encoded observations are decoded to new elements from the same class. This technique, which we call a Variational Homoencoder (VHE), produces a hierarchical latent variable model which better utilises latent variables. We use the VHE framework to learn a hierarchical PixelCNN on the Omniglot dataset, which outperforms all existing models on test set likelihood and achieves strong performance on one-shot generation and classification tasks. We additionally validate the VHE on natural images from the YouTube Faces database. Finally, we develop extensions of the model that apply to richer dataset structures such as factorial and hierarchical categories. Much research has focused on developing neural architectures for few-shot classification (Koch, 2015; Vinyals et al. , 2016; Snell et al. , 2017; Santoro et al. , 2016). These discriminatively-trained networks take as input a test example and a ‘support set’ of examples from several novel classes, and determine the most likely classification of the test example within the novel classes. A second approach, as explored in Ravi & Larochelle (2016); Finn et al. (2017), is to use only a standard classification network but adapt its parameters to the support examples with a learned initialisation and update rule. In either case, such discriminative models can achieve stateof-the-art few-shot classification performance, although they provide no principled means for transferring knowledge to other tasks.

Details