Firas Hamze

UAI Conference 2010 Conference Paper

Intracluster Moves for Constrained Discrete-Space MCMC

• Firas Hamze
• Nando de Freitas

This paper addresses the problem of sampling from binary distributions with constraints. In particular, it proposes an MCMC method to draw samples from a distribution of the set of all states at a specified distance from some reference state. For example, when the reference state is the vector of zeros, the algorithm can draw samples from a binary distribution with a constraint on the number of active variables, say the number of 1’s. We motivate the need for this algorithm with examples from statistical physics and probabilistic inference. Unlike previous algorithms proposed to sample from binary distributions with these constraints, the new algorithm allows for large moves in state space and tends to propose them such that they are energetically favourable. The algorithm is demonstrated on three Boltzmann machines of varying difficulty: A ferromagnetic Ising model (with positive potentials), a restricted Boltzmann machine with learned Gabor-like filters as potentials, and a challenging three-dimensional spin-glass (with positive and negative potentials).

NeurIPS Conference 2009 Conference Paper

A Stochastic approximation method for inference in probabilistic graphical models

• Peter Carbonetto
• Matthew King
• Firas Hamze

We describe a new algorithmic framework for inference in probabilistic models, and apply it to inference for latent Dirichlet allocation. Our framework adopts the methodology of variational inference, but unlike existing variational methods such as mean field and expectation propagation it is not restricted to tractable classes of approximating distributions. Our approach can also be viewed as a sequential Monte Carlo (SMC) method, but unlike existing SMC methods there is no need to design the artificial sequence of distributions. Notably, our framework offers a principled means to exchange the variance of an importance sampling estimate for the bias incurred through variational approximation. Experiments on a challenging inference problem in population genetics demonstrate improvements in stability and accuracy over existing methods, and at a comparable cost.

UAI Conference 2007 Conference Paper

Large-Flip Importance Sampling

• Firas Hamze
• Nando de Freitas

ICRA Conference 2005 Conference Paper

Fast Computational Methods for Visually Guided Robots

• Maryam Mahdaviani
• Nando de Freitas
• Bob Fraser
• Firas Hamze

This paper proposes numerical algorithms for reducing the computational cost of semi-supervised and active learning procedures for visually guided mobile robots from O(M 3 to O(M), while reducing the storage requirements from M 2 to M. This reduction in cost is essential for real-time interaction with mobile robots. The considerable speed ups are achieved using Krylov subspace methods and the fast Gauss transform. Although these state-of-the-art numerical algorithms are known, their application to semi-supervised learning, active learning and mobile robotics is new and should be of interest and great value to the robotics community. We apply our fast algorithms to interactive object recognition on Sony’s ERS-7 Aibo. We provide comparisons that clearly demonstrate remarkable improvements in computational speed.

NeurIPS Conference 2005 Conference Paper

Hot Coupling: A Particle Approach to Inference and Normalization on Pairwise Undirected Graphs

• Firas Hamze
• Nando de Freitas

This paper presents a new sampling algorithm for approximating func- tions of variables representable as undirected graphical models of arbi- trary connectivity with pairwise potentials, as well as for estimating the notoriously dif(cid: 2)cult partition function of the graph. The algorithm (cid: 2)ts into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of in- termediate distributions which get closer to the desired one. While the idea of using (cid: 147)tempered(cid: 148) proposals is known, we construct a novel se- quence of target distributions where, rather than dropping a global tem- perature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning tree of the variables. We present experimental results on inference and estimation of the parti- tion function for sparse and densely-connected graphs.

UAI Conference 2004 Conference Paper

From Fields to Trees

• Firas Hamze
• Nando de Freitas

We present new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also construct new information theory tools for comparing different MCMC schemes and show that, under these, tree sampling is more efficient.