Author name cluster

Thomas Hannagan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

TMLR Journal 2025 Journal Article

The Initialization Determines Whether In-Context Learning Is Gradient Descent

Shifeng Xie
Rui Yuan
Simone Rossi
Thomas Hannagan

In-context learning (ICL) in large language models (LLMs) is a striking phenomenon, yet its underlying mechanisms remain only partially understood. Previous work connects linear self-attention (LSA) to gradient descent (GD), this connection has primarily been established under simplified conditions with zero-mean Gaussian priors and zero initialization for GD. However, subsequent studies have challenged this simplified view by highlighting its overly restrictive assumptions, demonstrating instead that under conditions such as multi-layer or nonlinear attention, self-attention performs optimization-like inference, akin to but distinct from GD. We investigate how multi-head LSA approximates GD under more realistic conditions—specifically when incorporating non-zero Gaussian prior means in linear regression formulations of ICL. We first extend multi-head LSA embedding matrix by introducing an initial estimation of the query, referred to as the initial guess. We prove an upper bound on the number of heads needed for ICL linear regression setup. Our experiments confirm this result and further observe that a performance gap between one-step GD and multi-head LSA persists. To address this gap, we introduce $y_q$-LSA, a simple generalization of single-head LSA with a trainable initial guess $y_q$. We theoretically establish the capabilities of $y_q$-LSA and provide experimental validation on linear regression tasks, thereby extending the theory that bridges ICL and GD. Finally, inspired by our findings in the case of linear regression, we consider widespread LLMs augmented with initial guess capabilities, and show that their performance is improved on a semantic similarity task.

PDF Details

EWRL Workshop 2024 Workshop Paper

Unbiased Policy Gradient with Random Horizon

Rui Yuan
Andrii Tretynko
Simone Rossi
Thomas Hannagan

Policy gradient (PG) methods are widely used in reinforcement learning. However, for infinite-horizon discounted reward settings, practical implementations of PG usually must rely on biased gradient estimators, due to the truncated finite-horizon sampling, which limits actual performance and hinders theoretical analysis. In this work, we introduce a new family of algorithms, __unbiased policy gradient__ (UPG), that enables unbiased gradient estimators by considering finite-horizon undiscounted rewards, where the horizon is randomly sampled from a geometric distribution $\mathrm{Geom}(1-\gamma)$ associated to the discount factor $\gamma$. Thanks to the absence of bias, UPG achieves the $\mathcal{O}(\epsilon^{-4})$ sample complexity to a stationary point, which is improved by $\mathcal{O}(\log\epsilon^{-1})$, compared to the one of the vanilla PG, and is met with fewer assumptions. Our work also provides a new angle on well-known algorithms such as Q-PGT and RPG. We recover the unbiased Q-PGT algorithm as a special case of UPG, allowing for its first sample complexity analysis. We further show that UPG can be extended to $\alpha$-UPG, a more generic class of PG algorithms which performs unbiased gradient estimators and notably admits RPG as a special case. The general sample complexity analysis of $\alpha$-UPG that we present enables to recover the convergence rates of RPG, also with tighter bounds. Finally, we propose and evaluate two new algorithms within the UPG family: unbiased GPOMDP (UGPOMDP) and $\alpha$-UGPOMDP. We show theoretically and empirically on four different environments that both UGPOMDP and $\alpha$-UGPOMDP outperform its known vanilla PG counterpart, GPOMDP.

PDF

NeurIPS Conference 2023 Conference Paper

On permutation symmetries in Bayesian neural network posteriors: a variational perspective

Simone Rossi
Ankit Singh
Thomas Hannagan

The elusive nature of gradient-based optimization in neural networks is tied to their loss landscape geometry, which is poorly understood. However recent work has brought solid evidence that there is essentially no loss barrier between the local solutions of gradient descent, once accounting for weight-permutations that leave the network's computation unchanged. This raises questions for approximate inference in Bayesian neural networks (BNNs), where we are interested in marginalizing over multiple points in the loss landscape. In this work, we first extend the formalism of marginalized loss barrier and solution interpolation to BNNs, before proposing a matching algorithm to search for linearly connected solutions. This is achieved by aligning the distributions of two independent approximate Bayesian solutions with respect to permutation matrices. Building on the work of Ainsworth et al. (2023), we frame the problem as a combinatorial optimization one, using an approximation to the sum of bilinear assignment problem. We then experiment on a variety of architectures and datasets, finding nearly zero marginalized loss barriers for linearly connected solutions.

PDF Details