Author name cluster

Sang Wan Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

10 papers

1 author row

RLDM Conference 2025 Conference Abstract

RLDM 2025 Abstract Booklet 52 Deep Neural Experimenter: Enhancing Human One-shot Inference Through Adaptive Task Design Su Jin An† Changhwa Lee†

perimenter: Enhancing Human One-shot Inference
Through Adaptive Task Design
Su Jin An† Changhwa Lee†
Daejeon
Republic of Korea Daejeon
Republic of Korea
Sang Wan Lee
KAIST

Booklet 52 Deep Neural Experimenter: Enhancing Human One-shot Inference Through Adaptive Task Design Su Jin An† Changhwa Lee† Center for Neuroscience-inspired Artificial Intelligence Program of Brain and Cognitive Engineering Korea Advanced Institute of Science & Technology (KAIST) KAIST Daejeon, Republic of Korea Daejeon, Republic of Korea sujinan@kaist. ac. kr ckdghk77@kaist. ac. kr Sang Wan Lee Department of Brain and Cognitive Sciences Graduate School of Data Science Kim Jaechul Graduate School of AI Center for Neuroscience-inspired Artificial Intelligence KAIST Daejeon, Republic of Korea sangwan@kaist. ac. kr Abstract Model-based approaches in computational neuroscience have considerably advanced our understanding of human behaviour. Yet, their efficacy critically depends on the experimental tasks they are applied to—particularly on how well these tasks isolate and elicit the targeted cognitive processes, such as causal reasoning. Traditional open-loop paradigms, which rely on fixed, pre-scripted task sequences, often fail to accommodate real-time fluctuations in cognitive state or inter-individual variability. As a result, they limit learning efficiency, reduce ecological validity, and hinder interpretability. We introduce the Deep Neural Experimenter (DNE), a closed-loop, model-based framework that adaptively modifies task structure in real time based on individual behavioural feedback. DNE comprises two interacting modules: a Profiler, implemented via a Long Short-Term Memory (LSTM) network with attention mechanisms, which estimates latent cognitive states from stimulus-response histories, and a Controller, which dynamically adjusts task parameters (e. g. , cue ordering, feedback salience) to align participant behaviour with a target model of inference. This profiler–controller loop formally mirrors the structure of an actor-critic architecture in reinforcement learning: the profiler estimates the value of cognitive states (critic), while the controller selects optimal task adjustments (actor) to guide inference trajectories. To validate DNE, we adopted a human causal inference task encompassing both incremental and one-shot learning under varying levels of uncertainty. Participants were asked to infer probabilistic relationships between abstract stimuli and outcomes. Those guided by DNE exhibited a 42% improvement in one-shot inference accuracy compared to a control group exposed to static, open-loop task sequences. Moreover, the experimental group demonstrated accelerated learning rates, maintained robustness to cognitive biases (e. g. , primacy and recency) and exhibited sustained improvement across sessions. These findings demonstrate that closed-loop, adaptive task design not only enhances learning outcomes but also redefines the experimental paradigm itself—from passive observation to active cognitive modulation. DNE thus provides a generalisable foundation for applications in adaptive tutoring systems, neuroadaptive interfaces, and precision cognitive interventions.

PDF Details

AAAI Conference 2025 Conference Paper

Spectral Motion Alignment for Video Motion Transfer Using Diffusion Models

Geon Yeong Park
Hyeonho Jeong
Sang Wan Lee
Jong Chul Ye

Diffusion models have significantly facilitated the customization of input video with target appearance while maintaining its motion patterns. To distill the motion information from video frames, existing works often estimate motion representations as frame difference or correlation in pixel-/feature-space. Despite its simplicity, these methods have unexplored limitations, including lack of understanding of global motion context, and the introduction of motion-independent spatial distortions. To address this, we present Spectral Motion Alignment (SMA), a novel framework that refines and aligns motion representations in the spectral domain. Specifically, SMA learns spectral motion representations, facilitating the learning of whole-frame global motion dynamics, and effectively mitigating motion-independent artifacts. Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

Jeonghwan Cheon
Sang Wan Lee
Se-Bum Paik

The brain prepares for learning even before interacting with the environment, by refining and optimizing its structures through spontaneous neural activity that resembles random noise. However, the mechanism of such a process has yet to be understood, and it is unclear whether this process can benefit the algorithm of machine learning. Here, we study this issue using a neural network with a feedback alignment algorithm, demonstrating that pretraining neural networks with random noise increases the learning efficiency as well as generalization abilities without weight transport. First, we found that random noise training modifies forward weights to match backward synaptic feedback, which is necessary for teaching errors by feedback alignment. As a result, a network with pre-aligned weights learns notably faster and reaches higher accuracy than a network without random noise training, even comparable to the backpropagation algorithm. We also found that the effective dimensionality of weights decreases in a network pretrained with random noise. This pre-regularization allows the network to learn simple solutions of a low rank, reducing the generalization error during subsequent training. This also enables the network to robustly generalize a novel, out-of-distribution dataset. Lastly, we confirmed that random noise pretraining reduces the amount of meta-loss, enhancing the network ability to adapt to various tasks. Overall, our results suggest that random noise training with feedback alignment offers a straightforward yet effective method of pretraining that facilitates quick and reliable learning without weight transport.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Models

Geon Yeong Park
Jeongsol Kim
Beomsu Kim
Sang Wan Lee
Jong Chul Ye

Despite the remarkable performance of text-to-image diffusion models in image generation tasks, recent studies have raised the issue that generated images sometimes cannot capture the intended semantic contents of the text prompts, which phenomenon is often called semantic misalignment. To address this, here we present a novel energy-based model (EBM) framework for adaptive context control by modeling the posterior of context vectors. Specifically, we first formulate EBMs of latent image representations and text embeddings in each cross-attention layer of the denoising autoencoder. Then, we obtain the gradient of the log posterior of context vectors, which can be updated and transferred to the subsequent cross-attention layer, thereby implicitly minimizing a nested hierarchy of energy functions. Our latent EBMs further allow zero-shot compositional generation as a linear combination of cross-attention outputs from different contexts. Using extensive experiments, we demonstrate that the proposed method is highly effective in handling various image generation tasks, including multi-concept generation, text-guided image inpainting, and real and synthetic image editing. Code: https: //github. com/EnergyAttention/Energy-Based-CrossAttention.

PDF Details

AAAI Conference 2021 Conference Paper

Human Uncertainty Inference via Deterministic Ensemble Neural Networks

Yujin Cha
Sang Wan Lee

The estimation and inference of human predictive uncertainty have great potential to improve the sampling efficiency and prediction reliability of human-in-the-loop systems for smart healthcare, smart education, and human-computer interactions. Predictive uncertainty in humans is highly interpretable, but its measurement is poorly accessible. Contrarily, the predictive uncertainty of machine learning models, albeit with poor interpretability, is relatively easily accessible. Here, we demonstrate that the poor accessibility of human uncertainty can be resolved by exploiting simple and universally accessible deterministic neural networks. We propose a new model for human uncertainty inference, called proxy ensemble network (PEN). Simulations with a few benchmark datasets demonstrated that the model can efficiently learn human uncertainty from a small amount of data. To show its applicability in real-world problems, we performed behavioral experiments, in which 64 physicians classified medical images and reported their level of confidence. We showed that the PEN could predict both the uncertainty range and diagnoses given by subjects with high accuracy. Our results demonstrate the ability of machine learning in guiding human decision making; it can also help humans in learning more efficiently and accurately. To the best of our knowledge, this is the first study that explored the possibility of accessing human uncertainty via the lens of deterministic neural networks.

PDF Details

RLDM Conference 2019 Conference Abstract

Behavioral and neural evidence for intrinsic motivation effect on reinforce- ment learning

Dongjae Kim
Sang Wan Lee

Earlier studies showed that competition between model-based (MB) and model-free (MF) rein- forcement learning (RL) [1] is based on the recent history of prediction error about rewards (RPE) and states (SPE) [2]. One key assumption of these studies that the sensitivity to prediction error (PE) remains constant during task performance. However, the extent to which PE signals influences RL can vary over time. For example, non-zero PE can motivate the human to update her behavioral policy in one situation, but it does not necessarily motivate her to the same degree in other situations. Here we examine a new hypothesis that the brain’s sensitivity to zero PE functions as an intrinsic motivation for RL. For this we developed a new computational model in which the sensitivity to zero PE is incorporated into the arbitration control. By ap- plying our computational model to 82 subjects’ data, we found that our model accounts for subjects’ choice patterns significantly better than other models that do not take into account the intrinsic motivation effect. A subsequent model-based fMRI analysis revealed that the mean of zero-SPE distribution, which reflects the degree of SPE influence on the MB system, was found to correlate with neural activity in the lingual and fusiform gyrus. We also found neural evidence of interaction between this variable and the prediction reliability, the key variable for arbitration control [2], in the right inferior lateral and ventromedial prefrontal cortex, the brain region previously implicated in arbitration control and value integration, respectively. In the psychophysiological interactions analysis, we found that the intrinsically motivated MB system inhibits the interaction between ventromedial prefrontal cortex and posterior putamen, the area previously implicated in MF valuation [2], [3]. Taken together, our study provides behavioral and neural evidence of the effect of intrinsic motivation on arbitration control between MB and MF RL.

PDF Details

RLDM Conference 2019 Conference Abstract

Deciphering model-based and model-free reinforcement learning strategies and choices from EEG

Dongjae Kim
Sang Wan Lee

A decade of studies in decision making revealed that human behavior is explained by a mixed form of the two types of learning strategies: a model-based (MB) and a model-free (MF) reinforcement learning (RL) [1], [2]. Subsequent functional neuroimaging (fMRI) studies examined a prefrontal circuitry to arbitrate between the two learning [2], [3], placing the prefrontal cortex (PFC) as a meta-controller [2], [4]. Accumulating fMRI evidence showing the role of meta-control in human decision making raises expectation for directly reading out meta-control states using simple neural recordings with higher temporal resolution, such as EEG. Here we propose a novel decoding scheme, called a prefrontal meta-control decoder, which (1) learns from EEG signals to classify latent learning strategies (MB vs. MF) and (2) utilizes this information to further classify choice signals (left vs. right choice). In doing so, we first implemented a latent strategy decoder with 2D/3D convolutional neural networks (CNN), conditioned by the computational model of prefrontal meta-control developed in the previous fMRI study [2]. Second, by applying a class activation mapping (CAM) technique to this decoder, we found distinctive EEG signatures for each learning strategy. Lastly, we trained a choice signal decoder by hybridizing a Long short-term memory (LSTM) with the latent strategy decoder. The latent learning strategy decoder and the choice decoder showed very high classification accuracy (98% and 84%, respectively). To the best of our knowledge, this is the first study to examine the possibility of decoding latent learning strategies underlying decision making with high precision and of substantially improving performance of intention reading from EEG signals. Moreover, the EEG signatures of MB and MF RL found in the current study open up a new possibility for investigating brain dynamics underlying prefrontal meta-control during decision making.

PDF Details

RLDM Conference 2019 Conference Abstract

Designing model-based and model-free reinforcement learning tasks without human guidance

Jae Hoon Shin
Jee Hang Lee
Shuangyi Tong
Sang Hwan
Sang Wan Lee

Recent findings in decision neuroscience apprise of two different types of reinforcement learning (RL) to guide choice behavior: model-free RL and model-based RL. Task design to examine competition and interaction between these two RLs has remained a major challenge though a few studies appear to do so in specific settings. This effort is severely impeded by the fact that individual variability is high and an optimal combination of relevant task parameters is sensitive to context changes. To fully address these issues, here we propose a novel computational framework that learns an optimal task policy specifying a trial-by-trial configuration of task parameters in a way that maximally separates or correlates the two RL processes. Based on a dual agent setting, the framework exploits a game play between (i) an approximate human agent (e. g. , computational model of arbitration between model-based/model-free RL) whose goal is to interact with environment to maximize future returns and (ii) a task control agent whose goal is to drive the latent state of the approximate human agent to a desired state by deliberately manipulating task parameter values on a trial-by-trial basis. Large-scale simulations on 82 subjects’ data in 8 different scenarios (refer to Additional Details) show that the framework successfully learns an online task policy that optimally controls the estimated amount of prediction error of the approximate human agent. A subsequent post- hoc analysis revealed that the task policies in different scenarios have distinctively different task parameter configurations, each of which are well aligned to the objective of each scenario. Moreover, we found in a model permutation test that optimized task policies well reflect individual variability. The proposed framework is applicable to any RL task paradigm, and raises an optimistic expectation for optimal RL task design with a high exactitude of behavioral controllability.

PDF Details

RLDM Conference 2019 Conference Abstract

Metacognitive exploration in reinforcement learning

Su Jin An
Benedetto De Martino
Sang Wan Lee

Reinforcement learning (RL) theory explains how animals learn from experience. Empirical tests of RL heavily rely on simple task paradigms with a small number of options, limiting our understanding of the ability to explore an uncharted world with infinitely many options. We test a theoretical idea that metacognition, the ability to introspect and estimate one’s own level of uncertainty in the course of learning, shapes exploration during RL. By combining computational modeling with behavioral data obtained using a novel experiment (two-stage infinite-armed bandits), we provide the first evidence of human metacogni- tive exploration in RL. We found that uncertainty regarding environmental structure and reward prediction error guide arbitration between exploration and exploitation. Intriguingly, we also found that optimality of arbitration depends on the individual metacognitive ability measured using an independent perceptual task, suggesting a key role for metacognition in fostering an optimal exploration policy to resolve uncertainty regarding environmental and reward structures.

PDF Details

RLDM Conference 2017 Conference Abstract

The role of task complexity during arbitration between model-based and model-free reinforce- ment learning

Sang Wan Lee
John P. O’Doherty

The balance between model-based and model-free reinforcement learning (RL) is suggested to be governed by an arbitration process, in which the degree of relative control of the two RLs over behavior is flexibly adjusted. However, a major open question concerns how the arbitration process operates. That is what are the key variables used to drive arbitration? One variable suggested to play an important role is the amount of uncertainty associated with each model’s respective predictions [1], [2]. . Here we explore the contribution of another potential variable to the arbitration process: the complexity of the state-space. We provide behavioral and neural evidence for an effect of task complexity on arbitration within the area of ventrolateral prefrontal cortex, the same region recently implicated in arbitrating between the two RLs [2]. Our findings also demonstrate an asymmetry in the nature of arbitration control: task complexity functions to regulate the degree of control exclusively on the model-free system, as opposed to the model-based. Another intriguing finding is that an excessive increase in task complexity caused human participants to resort to a default bias toward being either model-based or model-free, varying across participants. This suggests that excessive cognitive demands imposed by an arduous task results in participants relying instead on a default strategy irrespective of other variables otherwise driving arbitration such as reliability.

PDF Details