Author name cluster

Alex Gaunt

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

ICLR Conference 2022 Conference Paper

Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond

Jonathan Godwin
Michael Schaarschmidt
Alex Gaunt
Alvaro Sanchez-Gonzalez
Yulia Rubanova
Petar Velickovic
James Kirkpatrick
Peter W. Battaglia

In this paper we show that simple noisy regularisation can be an effective way to address oversmoothing. We first argue that regularisers ad-dressing oversmoothing should both penalise node latent similarity and encourage meaningful node representations. From this observation we derive “Noisy Nodes”,a simple technique in which we corrupt the input graph with noise, and add a noise correcting node-level loss. The diverse node level loss encourages latent node diversity, and the denoising objective encourages graph manifold learning. Our regulariser applies well-studied methods in simple, straightforward ways which allow even generic architectures to overcome oversmoothing and achieve state of the art results on quantum chemistry tasks such as QM9 and Open Catalyst, and improve results significantly on Open Graph Benchmark (OGB) datasets. Our results suggest Noisy Nodes can serve as a complementary building block in the GNN toolkit.

Details

ICLR Conference 2019 Conference Paper

Deterministic Variational Inference for Robust Bayesian Neural Networks

Anqi Wu
Sebastian Nowozin
Edward Meeds
Richard E. Turner
José Miguel Hernández-Lobato
Alex Gaunt

Bayesian neural networks (BNNs) hold great promise as a flexible and principled solution to deal with uncertainty when learning from finite data. Among approaches to realize probabilistic inference in deep neural networks, variational Bayes (VB) is theoretically grounded, generally applicable, and computationally efficient. With wide recognition of potential advantages, why is it that variational Bayes has seen very limited practical use for BNNs in real applications? We argue that variational inference in neural networks is fragile: successful implementations require careful initialization and tuning of prior variances, as well as controlling the variance of Monte Carlo gradient estimates. We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances. Combining these two innovations, the resulting method is highly efficient and robust. On the application of heteroscedastic regression we demonstrate good predictive performance over alternative approaches.

Details

ICML Conference 2017 Conference Paper

Differentiable Programs with Neural Libraries

Alex Gaunt
Marc Brockschmidt
Nate Kushman
Daniel Tarlow

We develop a framework for combining differentiable programming languages with neural networks. Using this framework we create end-to-end trainable systems that learn to write interpretable algorithms with perceptual components. We explore the benefits of inductive biases for strong generalization and modularity that come from the program-like structure of our models. In particular, modularity allows us to learn a library of (neural) functions which grows and improves as more tasks are solved. Empirically, we show that this leads to lifelong learning systems that transfer knowledge to new tasks more effectively than baselines.

Details

UAI Conference 2016 Conference Paper

Training Neural Nets to Aggregate Crowdsourced Responses

Alex Gaunt
Diana Borsa
Yoram Bachrach

We propose a new method for aggregating crowdsourced responses, based on a deep neural network. Once trained, the aggregator network gets as input the responses of multiple participants to the same set of questions, and outputs its prediction for the correct response to each question. We empirically evaluate our approach on a dataset of responses to a standard IQ questionnaire, and show it outperforms existing state-ofthe-art methods.

Details