Author name cluster

Deunsol Yoon

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

AAAI Conference 2026 System Paper

RAPID: A Rapid Prototyping Platform for Industrial Automation

Sunghoon Hong
Junseok Park
Whiyoung Jung
Deunsol Yoon
Woohyung Lim
Soonyoung Lee
Kanghoon Lee

Industrial automation in smart logistics and factories requires simulation platforms that support rapid environment building before costly physical deployment. Yet existing tools often require substantial expertise, complex setup, and long configuration times, hindering agile prototyping. We present RAPID, a simulation platform with two components: layout design, which enables intuitive visual configuration of factory layouts, and behavior simulation and validation, which allows users to attach behavior models and evaluate system performance. RAPID lowers the entry barrier to industrial simulation, letting users apply existing behavior models or trained reinforcement learning (RL) agents to new layouts with minimal effort. This approach lets practitioners prototype facilities in minutes rather than weeks and gives researchers a standardized environment for benchmarking multi-agent RL and coordination algorithms. By combining rapid design with simulation-based validation, RAPID accelerates automation development from concept to implementation.

PDF Details DOI

AAAI Conference 2026 System Paper

RL-Studio: A System for Multi-Phase Reinforcement Learning Experimentation

Whiyoung Jung
Sunghoon Hong
Deunsol Yoon
Jeonghye Kim
Yongjae Shin
Suhyun Jung
Hyundam Yoo
Youngjin Kim

Reinforcement learning (RL) has evolved beyond monolithic training, yet existing frameworks remain limited to single algorithms or simple offline-to-online transitions. We present multi-phase RL, a framework that orchestrates multiple learning phases for continual policy improvement. It enables efficient fine-tuning of pretrained policies with new data and smooth adaptation from simulation to real-world environments. To support this paradigm, we introduce RL-Studio, a platform that addresses key implementation barriers, including neural architecture mismatches, parameter transfer complexities, and experiment management overhead. It provides phase orchestration, transition-point monitoring, and full experiment lineage tracking. We demonstrate the effectiveness of multi-phase RL through representative scenarios and highlight RL-Studio’s capabilities.

PDF Details DOI

ICML Conference 2025 Conference Paper

Agent-Centric Actor-Critic for Asynchronous Multi-Agent Reinforcement Learning

Whiyoung Jung
Sunghoon Hong
Deunsol Yoon
Kanghoon Lee
Woohyung Lim

Multi-Agent Reinforcement Learning (MARL) struggles with coordination in sparse reward environments. Macro-actions —sequences of actions executed as single decisions— facilitate long-term planning but introduce asynchrony, complicating Centralized Training with Decentralized Execution (CTDE). Existing CTDE methods use padding to handle asynchrony, risking misaligned asynchronous experiences and spurious correlations. We propose the Agent-Centric Actor-Critic (ACAC) algorithm to manage asynchrony without padding. ACAC uses agent-centric encoders for independent trajectory processing, with an attention-based aggregation module integrating these histories into a centralized critic for improved temporal abstractions. The proposed structure is trained via a PPO-based algorithm with a modified Generalized Advantage Estimation for asynchronous environments. Experiments show ACAC accelerates convergence and enhances performance over baselines in complex MARL tasks.

Details

ICML Conference 2025 Conference Paper

Online Pre-Training for Offline-to-Online Reinforcement Learning

Yongjae Shin
Jeonghye Kim
Whiyoung Jung
Sunghoon Hong
Deunsol Yoon
Youngsoo Jang
Geon-Hyeong Kim
Jongseong Chae

Offline-to-online reinforcement learning (RL) aims to integrate the complementary strengths of offline and online RL by pre-training an agent offline and subsequently fine-tuning it through online interactions. However, recent studies reveal that offline pre-trained agents often underperform during online fine-tuning due to inaccurate value estimation caused by distribution shift, with random initialization proving more effective in certain cases. In this work, we propose a novel method, Online Pre-Training for Offline-to-Online RL (OPT), explicitly designed to address the issue of inaccurate value estimation in offline pre-trained agents. OPT introduces a new learning phase, Online Pre-Training, which allows the training of a new value function tailored specifically for effective online fine-tuning. Implementation of OPT on TD3 and SPOT demonstrates an average 30% improvement in performance across a wide range of D4RL environments, including MuJoCo, Antmaze, and Adroit.

Details

ICML Conference 2025 Conference Paper

Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data

Jeonghye Kim
Yongjae Shin
Whiyoung Jung
Sunghoon Hong
Deunsol Yoon
Youngchul Sung
Kanghoon Lee
Woohyung Lim

Reinforcement learning with offline data suffers from Q-value extrapolation errors. To address this issue, we first demonstrate that linear extrapolation of the Q-function beyond the data range is particularly problematic. To mitigate this, we propose guiding the gradual decrease of Q-values outside the data range, which is achieved through reward scaling with layer normalization (RS-LN) and a penalization mechanism for infeasible actions (PA). By combining RS-LN and PA, we develop a new algorithm called PARS. We evaluate PARS across a range of tasks, demonstrating superior performance compared to state-of-the-art algorithms in both offline training and online fine-tuning on the D4RL benchmark, with notable success in the challenging AntMaze Ultra task.

Details

AAMAS Conference 2024 Conference Paper

Naphtha Cracking Center Scheduling Optimization using Multi-Agent Reinforcement Learning

Sunghoon Hong
Deunsol Yoon
Whiyoung Jung
Jinsang Lee
Hyundam Yoo
Jiwon Ham
Suhyun Jung
Chanwoo Moon

The Naphtha Cracking Center (NCC) is central to petrochemical feedstock production through the intricate process. It consists of receipt stage for unloading naphtha, blending stage for mixing naphtha, and furnace stage for producing marketable products. It is crucial to make an optimal schedule for NCC for profitability and efficiency. Traditionally managed by human experts, challenges arise in predicting complex chemical reactions and navigating real-world complexities. To address these issues, this paper aims to develop autonomous NCC operation using multi-agent reinforcement learning, where each agent is responsible for each stage and collaborates to achieve common objectives, while adhering to real-world constraints. We developed an online web service to allow the staff in LG Chem Daesan NCC facility to obtain an NCC schedule in real-time, and the staff are now operating the facility based on the schedules generated by the online web service.

PDF

ICLR Conference 2022 Conference Paper

Structure-Aware Transformer Policy for Inhomogeneous Multi-Task Reinforcement Learning

Sunghoon Hong
Deunsol Yoon
Kee-Eung Kim

Modular Reinforcement Learning, where the agent is assumed to be morphologically structured as a graph, for example composed of limbs and joints, aims to learn a policy that is transferable to a structurally similar but different agent. Compared to traditional Multi-Task Reinforcement Learning, this promising approach allows us to cope with inhomogeneous tasks where the state and action space dimensions differ across tasks. Graph Neural Networks are a natural model for representing the pertinent policies, but a recent work has shown that their multi-hop message passing mechanism is not ideal for conveying important information to other modules and thus a transformer model without morphological information was proposed. In this work, we argue that the morphological information is still very useful and propose a transformer policy model that effectively encodes such information. Specifically, we encode the morphological information in terms of the traversal-based positional embedding and the graph-based relational embedding. We empirically show that the morphological information is crucial for modular reinforcement learning, substantially outperforming prior state-of-the-art methods on multi-task learning as well as transfer learning settings with different state and action space dimensions.

Details

ICLR Conference 2021 Conference Paper

Winning the L2RPN Challenge: Power Grid Management via Semi-Markov Afterstate Actor-Critic

Deunsol Yoon
Sunghoon Hong
Byung-Jun Lee 0001
Kee-Eung Kim

Safe and reliable electricity transmission in power grids is crucial for modern society. It is thus quite natural that there has been a growing interest in the automatic management of power grids, exempliﬁed by the Learning to Run a Power Network Challenge (L2RPN), modeling the problem as a reinforcement learning (RL) task. However, it is highly challenging to manage a real-world scale power grid, mostly due to the massive scale of its state and action space. In this paper, we present an off-policy actor-critic approach that effectively tackles the unique challenges in power grid management by RL, adopting the hierarchical policy together with the afterstate representation. Our agent ranked ﬁrst in the latest challenge (L2RPN WCCI 2020), being able to avoid disastrous situations while maintaining the highest level of operational efﬁciency in every test scenarios. This paper provides a formal description of the algorithmic aspect of our approach, as well as further experimental studies on diverse power grids.

Details