Author name cluster

Alex Nguyen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ICLR Conference 2025 Conference Paper

Differential learning kinetics govern the transition from memorization to generalization during in-context learning

Alex Nguyen
Gautam Reddy

Transformers exhibit in-context learning (ICL): the ability to use novel information presented in the context without additional weight updates. Recent work shows that ICL emerges when models are trained on a sufficiently diverse set of tasks and the transition from memorization to generalization is sharp with increasing task diversity. One interpretation is that a network's limited capacity to memorize favors generalization. Here, we examine the mechanistic underpinnings of this transition using a small transformer applied to a synthetic ICL task. Using theory and experiment, we show that the sub-circuits that memorize and generalize can be viewed as largely independent. The relative *rates* at which these sub-circuits learn explains the transition from memorization to generalization, rather than capacity constraints. We uncover a memorization scaling law, which determines the task diversity threshold at which the network generalizes. The theory quantitatively explains a variety of other ICL-related phenomena, including the long-tailed distribution of when ICL is acquired, the bimodal behavior of solutions close to the task diversity threshold, the influence of contextual and data distributional statistics on ICL, and the transient nature of ICL.

Details

NeurIPS Conference 2025 Conference Paper

Generalization vs Specialization under Concept Shift

Alex Nguyen
David Schwab
Vudtiwat Ngampruetikorn

Machine learning models are often brittle under distribution shift, i. e. , when data distributions at test time differ from those during training. Understanding this failure mode is central to identifying and mitigating safety risks of mass adoption of machine learning. Here we analyze ridge regression under concept shift—a form of distribution shift in which the input-label relationship changes at test time. We derive an exact expression for prediction risk in the thermodynamic limit. Our results reveal nontrivial effects of concept shift on generalization performance, including a phase transition between weak and strong concept shift regimes and nonmonotonic data dependence of test performance even when double descent is absent. Our theoretical results are in good agreement with experiments based on transformers pretrained to solve linear regression; under concept shift, too long context length can be detrimental to generalization performance of next token prediction. Finally, experiments on MNIST and FashionMNIST further validate our theoretical predictions, suggesting these phenomena represent a fundamental aspect of learning under distribution shift.

PDF Details

NeurIPS Conference 2023 Conference Paper

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

Paul Scotti
Atmadeep Banerjee
Jimmie Goode
Stepan Shabalin
Alex Nguyen
ethan cohen
Aidan Dempster
Nathalie Verlinde

We present MindEye, a novel fMRI-to-image approach to retrieve and reconstruct viewed images from brain activity. Our model comprises two parallel submodules that are specialized for retrieval (using contrastive learning) and reconstruction (using a diffusion prior). MindEye can map fMRI brain activity to any high dimensional multimodal latent space, like CLIP image space, enabling image reconstruction using generative models that accept embeddings from this latent space. We comprehensively compare our approach with other existing methods, using both qualitative side-by-side comparisons and quantitative evaluations, and show that MindEye achieves state-of-the-art performance in both reconstruction and retrieval tasks. In particular, MindEye can retrieve the exact original image even among highly similar candidates indicating that its brain embeddings retain fine-grained image-specific information. This allows us to accurately retrieve images even from large-scale databases like LAION-5B. We demonstrate through ablations that MindEye's performance improvements over previous methods result from specialized submodules for retrieval and reconstruction, improved training techniques, and training models with orders of magnitude more parameters. Furthermore, we show that MindEye can better preserve low-level image features in the reconstructions by using img2img, with outputs from a separate autoencoder. All code is available on GitHub.

PDF Details

YNIMG Journal 2021 Journal Article

Cortical gradients of functional connectivity are robust to state-dependent changes following sleep deprivation

Nathan Cross
Casey Paquola
Florence B. Pomares
Aurore A. Perrault
Aude Jegou
Alex Nguyen
Umit Aydin
Boris C. Bernhardt

Sleep deprivation leads to significant impairments in cognitive performance and changes to the interactions between large scale cortical networks, yet the hierarchical organization of cortical activity across states is still being explored. We used functional magnetic resonance imaging to assess activations and connectivity during cognitive tasks in 20 healthy young adults, during three states: (i) following a normal night of sleep, (ii) following 24hr of total sleep deprivation, and (iii) after a morning recovery nap. Situating cortical activity during cognitive tasks along hierarchical organizing gradients based upon similarity of functional connectivity patterns, we found that regional variations in task-activations were captured by an axis differentiating areas involved in executive control from default mode regions and paralimbic cortex. After global signal regression, the range of functional differentiation along this axis at baseline was significantly related to decline in working memory performance (2-back task) following sleep deprivation, as well as the extent of recovery in performance following a nap. The relative positions of cortical regions within gradients did not significantly change across states, except for a lesser differentiation of the visual system and increased coupling of the posterior cingulate cortex with executive control areas after sleep deprivation. This was despite a widespread increase in the magnitude of functional connectivity across the cortex following sleep deprivation. Cortical gradients of functional differentiation thus appear relatively insensitive to state-dependent changes following sleep deprivation and recovery, suggesting that there are no large-scale changes in cortical functional organization across vigilance states. Certain features of particular gradient axes may be informative for the extent of decline in performance on more complex tasks following sleep deprivation, and could be beneficial over traditional voxel- or parcel-based approaches in identifying realtionships between state-dependent brain activity and behavior.

Details DOI

IJCAI Conference 2020 Conference Paper

RLCard: A Platform for Reinforcement Learning in Card Games

Daochen Zha
Kwei-Herng Lai
Songyi Huang
Yuanpu Cao
Keerthana Reddy
Juan Vargas
Alex Nguyen
Ruzhe Wei

We present RLCard, a Python platform for reinforcement learning research and development in card games. RLCard supports various card environments and several baseline algorithms with unified easy-to-use interfaces, aiming at bridging reinforcement learning and imperfect information games. The platform provides flexible configurations of state representation, action encoding, and reward design. RLCard also supports visualizations for algorithm debugging. In this demo, we showcase two representative environments and their visualization results. We conclude this demo with challenges and research opportunities brought by RLCard. A video is available on YouTube.

PDF Details DOI