Author name cluster

Claudia Clopath

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

ICLR Conference 2025 Conference Paper

SIMPL: Scalable and hassle-free optimisation of neural representations from behaviour

Tom M. George
Pierre Glaser
Kimberly L. Stachenfeld
Caswell Barry
Claudia Clopath

Neural activity in the brain is known to encode low-dimensional, time-evolving, behaviour-related variables. A long-standing goal of neural data analysis has been to identify these variables and their mapping to neural activity. A productive and canonical approach has been to simply visualise neural "tuning curves" as a function of behaviour. However, significant discrepancies between behaviour and the true latent variables -- such as an agent thinking of position Y whilst located at position X -- distort and blur the tuning curves, decreasing their interpretability. To address this, latent variable models propose to learn the latent variable from data; these are typically expensive, hard to tune, or scale poorly, complicating their adoption. Here we propose SIMPL (Scalable Iterative Maximization of Population-coded Latents), an EM-style algorithm which iteratively optimises latent variables and tuning curves. SIMPL is fast, scalable and exploits behaviour as an initial condition to further improve convergence and identifiability. It can accurately recover latent variables in spatial and non-spatial tasks. When applied to a large hippocampal dataset SIMPL converges on smaller, more numerous, and more uniformly sized place fields than those based on behaviour, suggesting the brain may encode space with greater resolution than previously thought.

Details

RLJ Journal 2025 Journal Article

Uncertainty Prioritized Experience Replay

Rodrigo Antonio Carrasco-Davis
Sebastian Lee
Claudia Clopath
Will Dabney

Prioritized experience replay, which improves sample efficiency by selecting relevant transitions to update parameter estimates, is a crucial component of contemporary value-based deep reinforcement learning models. Typically, transitions are prioritized based on their temporal difference error. However, this approach is prone to favoring noisy transitions, even when the value estimation closely approximates the target mean. This phenomenon resembles the *noisy TV* problem postulated in the exploration literature, in which exploration-guided agents get stuck by mistaking noise for novelty. To mitigate the disruptive effects of noise in value estimation, we propose using epistemic uncertainty estimation to guide the prioritization of transitions from the replay buffer. Epistemic uncertainty quantifies the uncertainty that can be reduced by learning, hence reducing transitions sampled from the buffer generated by unpredictable random processes. We first illustrate the benefits of epistemic uncertainty prioritized replay in two tabular toy models: a simple multi-arm bandit task, and a noisy gridworld. Subsequently, we evaluate our prioritization scheme on the Atari suite, outperforming quantile regression deep Q-learning benchmarks; thus forging a path for the use of uncertainty prioritized replay in reinforcement learning agents.

PDF Details

RLC Conference 2025 Conference Paper

Uncertainty Prioritized Experience Replay

Rodrigo Antonio Carrasco-Davis
Sebastian Lee
Claudia Clopath
Will Dabney

PDF Details

NeurIPS Conference 2024 Conference Paper

Feedback control guides credit assignment in recurrent neural networks

Klara Kaleb
Barbara Feulner
Juan A. Gallego
Claudia Clopath

How do brain circuits learn to generate behaviour? While significant strides have been made in understanding learning in artificial neural networks, applying this knowledge to biological networks remains challenging. For instance, while backpropagation is known to perform accurate credit assignment of error in artificial neural networks, how a similarly powerful process can be realized within the constraints of biological circuits remains largely unclear. One of the major challenges is that the brain's extensive recurrent connectivity requires the propagation of error through both space and time, a problem that is notoriously difficult to solve in vanilla recurrent neural networks. Moreover, the extensive feedback connections in the brain are known to influence forward network activity, but the interaction between feedback-driven activity changes and local, synaptic plasticity-based learning is not fully understood. Building on our previous work modelling motor learning, this work investigates the mechanistic properties of pre-trained networks with feedback control on a standard motor task. We show that feedback control of the ongoing recurrent network dynamics approximates the optimal first-order gradient with respect to the network activities, allowing for rapid, ongoing movement correction. Moreover, we show that trial-by-trial adaptation to a persistent perturbation using a local, biologically plausible learning rule that integrates recent activity and error feedback is both more accurate and more efficient with feedback control during learning, due to the decoupling of the recurrent network dynamics and the injection of an adaptive, second-order gradient into the network dynamics. Thus, our results suggest that feedback control may guide credit assignment in biological recurrent neural networks, enabling both rapid and efficient learning in the brain.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

A generative model of the hippocampal formation trained with theta driven local learning rules

Tom M George
Kimberly L. Stachenfeld
Caswell Barry
Claudia Clopath
Tomoki Fukai

Advances in generative models have recently revolutionised machine learning. Meanwhile, in neuroscience, generative models have long been thought fundamental to animal intelligence. Understanding the biological mechanisms that support these processes promises to shed light on the relationship between biological and artificial intelligence. In animals, the hippocampal formation is thought to learn and use a generative model to support its role in spatial and non-spatial memory. Here we introduce a biologically plausible model of the hippocampal formation tantamount to a Helmholtz machine that we apply to a temporal stream of inputs. A novel component of our model is that fast theta-band oscillations (5-10 Hz) gate the direction of information flow throughout the network, training it akin to a high-frequency wake-sleep algorithm. Our model accurately infers the latent state of high-dimensional sensory environments and generates realistic sensory predictions. Furthermore, it can learn to path integrate by developing a ring attractor connectivity structure matching previous theoretical proposals and flexibly transfer this structure between environments. Whereas many models trade-off biological plausibility with generality, our model captures a variety of hippocampal cognitive functions under one biologically plausible local learning rule.

PDF Details

ICML Conference 2022 Conference Paper

Maslow's Hammer in Catastrophic Forgetting: Node Re-Use vs. Node Activation

Sebastian Lee
Stefano Sarao Mannelli
Claudia Clopath
Sebastian Goldt
Andrew M. Saxe

Continual learning—learning new tasks in sequence while maintaining performance on old tasks—remains particularly challenging for artificial neural networks. Surprisingly, the amount of forgetting does not increase with the dissimilarity between the learned tasks, but appears to be worst in an intermediate similarity regime. In this paper we theoretically analyse both a synthetic teacher-student framework and a real data setup to provide an explanation of this phenomenon that we name Maslow’s Hammer hypothesis. Our analysis reveals the presence of a trade-off between node activation and node re-use that results in worst forgetting in the intermediate regime. Using this understanding we reinterpret popular algorithmic interventions for catastrophic interference in terms of this trade-off, and identify the regimes in which they are most effective.

Details

ICML Conference 2021 Conference Paper

Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective

Florin Gogianu
Tudor Berariu
Mihaela Rosca
Claudia Clopath
Lucian Busoniu
Razvan Pascanu

Most of the recent deep reinforcement learning advances take an RL-centric perspective and focus on refinements of the training objective. We diverge from this view and show we can recover the performance of these developments not by changing the objective, but by regularising the value-function estimator. Constraining the Lipschitz constant of a single layer using spectral normalisation is sufficient to elevate the performance of a Categorical-DQN agent to that of a more elaborated agent on the challenging Atari domain. We conduct ablation studies to disentangle the various effects normalisation has on the learning dynamics and show that is sufficient to modulate the parameter updates to recover most of the performance of spectral normalisation. These findings hint towards the need to also focus on the neural component and its learning dynamics to tackle the peculiarities of Deep Reinforcement Learning.

Details

ICML Conference 2019 Conference Paper

Policy Consolidation for Continual Reinforcement Learning

Christos Kaplanis
Murray Shanahan
Claudia Clopath

We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is agnostic to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries and can adapt in continuously changing environments. In our policy consolidation model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent’s policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

Details

ICML Conference 2018 Conference Paper

Continual Reinforcement Learning with Complex Synapses

Christos Kaplanis
Murray Shanahan
Claudia Clopath

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to abrupt erasure of previously acquired knowledge. Whereas in a neural network the parameters are typically modelled as scalar values, an individual synapse in the brain comprises a complex network of interacting biochemical components that evolve at different timescales. In this paper, we show that by equipping tabular and deep reinforcement learning agents with a synaptic model that incorporates this biological complexity (Benna & Fusi, 2016), catastrophic forgetting can be mitigated at multiple timescales. In particular, we find that as well as enabling continual learning across sequential training of two simple tasks, it can also be used to overcome within-task forgetting by reducing the need for an experience replay database.

Details

YNIMG Journal 2016 Journal Article

Local inhibitory plasticity tunes macroscopic brain dynamics and allows the emergence of functional brain networks

Peter J. Hellyer
Barbara Jachs
Claudia Clopath
Robert Leech

Rich, spontaneous brain activity has been observed across a range of different temporal and spatial scales. These dynamics are thought to be important for efficient neural functioning. A range of experimental evidence suggests that these neural dynamics are maintained across a variety of different cognitive states, in response to alterations of the environment and to changes in brain configuration (e. g. , across individuals, development and in many neurological disorders). This suggests that the brain has evolved mechanisms to maintain rich dynamics across a broad range of situations. Several mechanisms based around homeostatic plasticity have been proposed to explain how these dynamics emerge from networks of neurons at the microscopic scale. Here we explore how a homeostatic mechanism may operate at the macroscopic scale: in particular, focusing on how it interacts with the underlying structural network topology and how it gives rise to well-described functional connectivity networks. We use a simple mean-field model of the brain, constrained by empirical white matter structural connectivity where each region of the brain is simulated using a pool of excitatory and inhibitory neurons. We show, as with the microscopic work, that homeostatic plasticity regulates network activity and allows for the emergence of rich, spontaneous dynamics across a range of brain configurations, which otherwise show a very limited range of dynamic regimes. In addition, the simulated functional connectivity of the homeostatic model better resembles empirical functional connectivity network. To accomplish this, we show how the inhibitory weights adapt over time to capture important graph theoretic properties of the underlying structural network. Therefore, this work presents suggests how inhibitory homeostatic mechanisms facilitate stable macroscopic dynamics to emerge in the brain, aiding the formation of functional connectivity networks.

Details DOI

NeurIPS Conference 2007 Conference Paper

An online Hebbian learning rule that performs Independent Component Analysis

Claudia Clopath
André Longtin
Wulfram Gerstner

Independent component analysis (ICA) is a powerful method to decouple signals. Most of the algorithms performing ICA do not consider the temporal correlations of the signal, but only higher moments of its amplitude distribution. Moreover, they require some preprocessing of the data (whitening) so as to remove second order correlations. In this paper, we are interested in understanding the neural mechanism responsible for solving ICA. We present an online learning rule that exploits delayed correlations in the input. This rule performs ICA by detecting joint variations in the firing rates of pre- and postsynaptic neurons, similar to a local rate-based Hebbian learning rule.

PDF Details