Author name cluster

Chung-Wei Lin

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

ICML Conference 2025 Conference Paper

Directly Forecasting Belief for Reinforcement Learning with Delays

Qingyuan Wu
Yuhui Wang 0004
Simon Sinong Zhan
Yixuan Wang 0001
Chung-Wei Lin
Chen Lv 0001
Qi Zhu 0002
Jürgen Schmidhuber

Reinforcement learning (RL) with delays is challenging as sensory perceptions lag behind the actual events: the RL agent needs to estimate the real state of its environment based on past observations. State-of-the-art (SOTA) methods typically employ recursive, step-by-step forecasting of states. This can cause the accumulation of compounding errors. To tackle this problem, our novel belief estimation method, named Directly Forecasting Belief Transformer (DFBT), directly forecasts states from observations without incrementally estimating intermediate states step-by-step. We theoretically demonstrate that DFBT greatly reduces compounding errors of existing recursively forecasting methods, yielding stronger performance guarantees. In experiments with D4RL offline datasets, DFBT reduces compounding errors with remarkable prediction accuracy. DFBT’s capability to forecast state sequences also facilitates multi-step bootstrapping, thus greatly improving learning efficiency. On the MuJoCo benchmark, our DFBT-based method substantially outperforms SOTA baselines. Code is available at https: //github. com/QingyuanWuNothing/DFBT.

Details

IROS Conference 2025 Conference Paper

Landing-Aware Multi-Drone Routing in Last-Mile Delivery Services

JiHyun Kwon
Yi-Ying Chen
GaHyun Lee
Chung-Wei Lin
BaekGyu Kim

We propose a framework to compute the optimal routes for multi-drones to minimize the delivery time in the last-mile delivery service. We mainly focus on a notion of the landing exclusion zone that appears during the landing phase; an area around the drop-off site is blocked until a drop-off is completed. Such zones affect the delivery time as other drones need to detour or hover around the site unnecessarily. We formulate the Mixed-Integer Linear Programming (MILP) problem by explicitly modeling the landing phase. Then, we present the heuristic algorithm that iteratively solves a sequence of single-drone delivery problems according to the delivery priorities. A delivery priority is determined according to the spatiotemporal occupancy that quantifies the significance of the size of the landing exclusion zone and its blocking period. We designed the experiment for 48 urban delivery scenarios with varying density and distribution of delivery destinations, departure points, and order quantities. Our experiment results show that the heuristic computes the routes significantly faster than the original MILP, and the delivery time is 5% higher from the optimal solution (lower-bound), and 60% lower from the general requirement of a single package per round-trip (upper-bound).

Details

ICML Conference 2024 Conference Paper

Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays

Qingyuan Wu
Simon Sinong Zhan
Yixuan Wang 0001
Yuhui Wang 0004
Chung-Wei Lin
Chen Lv 0001
Qi Zhu 0002
Jürgen Schmidhuber

Reinforcement learning (RL) is challenging in the common case of delays between events and their sensory perceptions. State-of-the-art (SOTA) state augmentation techniques either suffer from state space explosion or performance degeneration in stochastic environments. To address these challenges, we present a novel Auxiliary-Delayed Reinforcement Learning (AD-RL) method that leverages auxiliary tasks involving short delays to accelerate RL with long delays, without compromising performance in stochastic environments. Specifically, AD-RL learns a value function for short delays and uses bootstrapping and policy improvement techniques to adjust it for long delays. We theoretically show that this can greatly reduce the sample complexity. On deterministic and stochastic benchmarks, our method significantly outperforms the SOTAs in both sample efficiency and policy performance. Code is available at https: //github. com/QingyuanWuNothing/AD-RL.

Details

NeurIPS Conference 2024 Conference Paper

Variational Delayed Policy Optimization

Qingyuan Wu
Simon S. Zhan
Yixuan Wang
Yuhui Wang
Chung-Wei Lin
Chen Lv
Qi Zhu
Chao Huang

In environments with delayed observation, state augmentation by including actions within the delay window is adopted to retrieve Markovian property to enable reinforcement learning (RL). Whereas, state-of-the-art (SOTA) RL techniques with Temporal-Difference (TD) learning frameworks commonly suffer from learning inefficiency, due to the significant expansion of the augmented state space with the delay. To improve the learning efficiency without sacrificing performance, this work novelly introduces Variational Delayed Policy Optimization (VDPO), reforming delayed RL as a variational inference problem. This problem is further modelled as a two-step iterative optimization problem, where the first step is TD learning in the delay-free environment with a small state space, and the second step is behaviour cloning which can be addressed much more efficiently than TD learning. We not only provide a theoretical analysis of VDPO in terms of sample complexity and performance, but also empirically demonstrate that VDPO can achieve consistent performance with SOTA methods, with a significant enhancement of sample efficiency (approximately 50\% less amount of samples) in the MuJoCo benchmark.

PDF Details DOI