Author name cluster

Stephanie Chan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

1 author row

NeurIPS Conference 2025 Conference Paper

The emergence of sparse attention: impact of data distribution and benefits of repetition

Nicolas Zucchet
Francesco D'Angelo
Andrew Lampinen
Stephanie Chan

Emergence is a fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden ways. Despite initial studies, we still lack a comprehensive understanding of how and when these abilities emerge. To address this gap, we study the emergence over training of sparse attention, a critical and frequently observed attention pattern in Transformers. By combining theoretical analysis of a toy model with empirical observations on small Transformers trained on a linear regression variant, we uncover the mechanics driving sparse attention emergence and reveal that emergence timing follows power laws based on task structure, architecture, and optimizer choice. We additionally find that repetition can greatly speed up emergence. Finally, we confirm these results on a well-studied in-context associative recall task. Our findings provide a simple, theoretically grounded framework for understanding how data distributions and model design influence the learning dynamics behind one form of emergence.

PDF Details

NeurIPS Conference 2024 Conference Paper

Many-Shot In-Context Learning

Rishabh Agarwal
Avi Singh
Lei Zhang
Bernd Bohnet
Luis Rosias
Stephanie Chan
Biao Zhang
Ankesh Anand

Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples – the many-shot regime. Going from few-shot to many-shot, we observe significant performance gains across a wide variety of generative and discriminative tasks. While promising, many-shot ICL can be bottlenecked by the available amount of human-generated outputs. To mitigate this limitation, we explore two new settings: (1) "Reinforced ICL" that uses model-generated chain-of-thought rationales in place of human rationales, and (2) "Unsupervised ICL" where we remove rationales from the prompt altogether, and prompts the model only with domain-specific inputs. We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. We demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases, can learn high-dimensional functions with numerical inputs, and performs comparably to supervised fine-tuning. Finally, we reveal the limitations of next-token prediction loss as an indicator of downstream ICL performance.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Passive learning of active causal strategies in agents and language models

Andrew Lampinen
Stephanie Chan
Ishita Dasgupta
Andrew Nam
Jane Wang

What can be learned about causality and experimentation from passive data? This question is salient given recent successes of passively-trained language models in interactive domains such as tool use. Passive learning is inherently limited. However, we show that purely passive learning can in fact allow an agent to learn generalizable strategies for determining and using causal structures, as long as the agent can intervene at test time. We formally illustrate that learning a strategy of first experimenting, then seeking goals, can allow generalization from passive learning in principle. We then show empirically that agents trained via imitation on expert data can indeed generalize at test time to infer and use causal links which are never present in the training data; these agents can also generalize experimentation strategies to novel variable sets never observed in training. We then show that strategies for causal intervention and exploitation can be generalized from passive data even in a more complex environment with high-dimensional observations, with the support of natural language explanations. Explanations can even allow passive learners to generalize out-of-distribution from perfectly-confounded training data. Finally, we show that language models, trained only on passive next-word prediction, can generalize causal intervention strategies from a few-shot prompt containing explanations and reasoning. These results highlight the surprising power of passive learning of active causal strategies, and have implications for understanding the behaviors and capabilities of language models.

PDF Details

NeurIPS Conference 2023 Conference Paper

The Transient Nature of Emergent In-Context Learning in Transformers

Aaditya Singh
Stephanie Chan
Ted Moskovitz
Erin Grant
Andrew Saxe
Felix Hill

Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e. g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL and in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to ``overtrain'' transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks. Finally, we present initial evidence that ICL transience may be caused by competition between ICL and IWL circuits.

PDF Details

NeurIPS Conference 2022 Conference Paper

Data Distributional Properties Drive Emergent In-Context Learning in Transformers

Stephanie Chan
Adam Santoro
Andrew Lampinen
Jane Wang
Aaditya Singh
Pierre Richemond
James McClelland
Felix Hill

Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the training regime lead to this emergent behavior? Here, we show that this behavior is driven by the distributions of the training data itself. In-context learning emerges when the training data exhibits particular distributional properties such as burstiness (items appear in clusters rather than being uniformly distributed over time) and having a large number of rarely occurring classes. In-context learning also emerges more strongly when item meanings or interpretations are dynamic rather than fixed. These properties are exemplified by natural language, but are also inherent to naturalistic data in a wide range of other domains. They also depart significantly from the uniform, i. i. d. training distributions typically used for standard supervised learning. In our initial experiments, we found that in-context learning traded off against more conventional weight-based learning, and models were unable to achieve both simultaneously. However, our later experiments uncovered that the two modes of learning could co-exist in a single model when it was trained on data following a skewed Zipfian distribution -- another common property of naturalistic data, including language. In further experiments, we found that naturalistic data distributions were only able to elicit in-context learning in transformers, and not in recurrent models. Our findings indicate how the transformer architecture works together with particular properties of the training data to drive the intriguing emergent in-context learning behaviour of large language models, and indicate how future work might encourage both in-context and in-weights learning in domains beyond language.

PDF Details

NeurIPS Conference 2022 Conference Paper

Semantic Exploration from Language Abstractions and Pretrained Representations

Allison Tam
Neil Rabinowitz
Andrew Lampinen
Nicholas A. Roy
Stephanie Chan
DJ Strouse
Jane Wang
Andrea Banino

Effective exploration is a challenge in reinforcement learning (RL). Novelty-based exploration methods can suffer in high-dimensional state spaces, such as continuous partially-observable 3D environments. We address this challenge by defining novelty using semantically meaningful state abstractions, which can be found in learned representations shaped by natural language. In particular, we evaluate vision-language representations, pretrained on natural image captioning datasets. We show that these pretrained representations drive meaningful, task-relevant exploration and improve performance on 3D simulated environments. We also characterize why and how language provides useful abstractions for exploration by considering the impacts of using representations from a pretrained model, a language oracle, and several ablations. We demonstrate the benefits of our approach with on- and off-policy RL algorithms and in two very different task domains---one that stresses the identification and manipulation of everyday objects, and one that requires navigational exploration in an expansive world. Our results suggest that using language-shaped representations could improve exploration for various algorithms and agents in challenging environments.

PDF Details

NeurIPS Conference 2021 Conference Paper

Towards mental time travel: a hierarchical memory for reinforcement learning agents

Andrew Lampinen
Stephanie Chan
Andrea Banino
Felix Hill

Reinforcement learning agents often forget details of the past, especially after delays or distractor tasks. Agents with common memory architectures struggle to recall and integrate across multiple timesteps of a past event, or even to recall the details of a single timestep that is followed by distractor tasks. To address these limitations, we propose a Hierarchical Chunk Attention Memory (HCAM), that helps agents to remember the past in detail. HCAM stores memories by dividing the past into chunks, and recalls by first performing high-level attention over coarse summaries of the chunks, and then performing detailed attention within only the most relevant chunks. An agent with HCAM can therefore "mentally time-travel"--remember past events in detail without attending to all intervening events. We show that agents with HCAM substantially outperform agents with other memory architectures at tasks requiring long-term recall, retention, or reasoning over memory. These include recalling where an object is hidden in a 3D environment, rapidly learning to navigate efficiently in a new neighborhood, and rapidly learning and retaining new words. Agents with HCAM can extrapolate to task sequences much longer than they were trained on, and can even generalize zero-shot from a meta-learning setting to maintaining knowledge across episodes. HCAM improve agent sample efficiency, generalization, and generality (by solving tasks that previously required specialized architectures). Our work is a step towards agents that can learn, interact, and adapt in complex and temporally-extended environments.

PDF Details

RLDM Conference 2015 Conference Abstract

Neural representations of posterior distributions over latent causes

Stephanie Chan
Kenneth Norman
Yael Niv

The world is governed by unobserved ‘causes’, which generate the events that we do observe. In reinforcement learning, and in particular, in partially observable Markov decision processes (POMDPs), these hidden causes are the ‘states’ of the task. Accurate inference about the current state, based on the agent’s observations, is critical for optimal decision making and learning. Here we investigate the neural basis of this type of inference about hidden causes in the human brain. In particular, we are interested in the neural substrates that allow humans to maintain, approximately or exactly, a belief distribution over the hidden states, which assigns varying levels of probability to each. We conducted an experiment in which participants viewed sequences of animals drawn from one of four ‘sectors’ in a safari. They were tasked with guessing which sector the animals were from, based on previous experience with the likelihood of each animal in each sector. We used functional magnetic resonance imaging (fMRI) to investigate brain representations of the posterior distribution P(sector — animals). Our results suggest that neural patterns in the lateral orbitofrontal cortex, angular gyrus, and precuneus correspond to a posterior distribution over ‘sectors’. We also show that a complementary set of areas are involved in the updating of the posterior distribution. These results are consistent with previous work implicating these areas in the representation of ‘state’ from reinforcement learning (Wilson et al, 2014) and ‘schemas’ or ‘situation models’ (Ranganath & Ritchey, 2012).

PDF Details

RLDM Conference 2013 Conference Abstract

“Identity prediction errors” and model-based learning

Stephanie Chan
Nina Lopatina
Yael Niv

It is known that humans and animals perform model-based reinforcement learning, in which decision-making uses a full model of the environment, including the transition probabilities between states. This is in contrast to model-free reinforcement learning, which relies on a “value” for each state or state- action pair. In model-free reinforcement learning, state values are thought to be learned using “value predic- tion errors” the difference between the expected value and the actual value observed. This is a computational- ly efficient means of learning, and, furthermore, these predictions errors famously seem to be represented by the activity of midbrain dopamine neurons. How are the full models, needed for model based reinforcement learning, learned? It has been hypothesized that there exist analogous “identity (state) prediction errors” in the brain, which are used to learn the transition probabilities between states. Identity prediction errors are elicited when a specific outcome (or state) is unexpected, regardless of whether the utility value of the outcome is different from what was expected. Here, we use fMRI to find signals corresponding to identity prediction errors in the brain. Based on prior work, we expected to find such signals in the orbitofrontal cortex.

PDF Details