Author name cluster

Kaiqiang Ke

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Reward-Free Action Poisoning in Offline RL via Conditional Shapley Value Estimation

Shenghong He
Chao Yu
Kaiqiang Ke
Yinqi Wei

Recentstudiesshowthatdatapoisoningattackscandegradetheperformance of offline reinforcement learning (RL) policies by strategically tampering with training datasets. However, existing methods generally assume the acquisition of reward signals during the generation of poisoning data, thus limiting their applications in real-world scenarios when the reward signals are not available. In this paper, we propose a novel method, called Shapley Action- Poisoning Attack (SAPA), which calculates the contribution of each state-actionpairinatrajectorytoidentifykeyactionsforpoisoning attacks without dependence on reward signals. Theoretical analysis proves that SAPA can degrade the performance of the learned policy below a specified threshold by tampering with key actions. Numerous experimental results demonstrate that SAPA surpasses state-of-the-artpoisoningmethodsintheattackperformanceunder various offline algorithms.

PDF

ICML Conference 2025 Conference Paper

Conservative Offline Goal-Conditioned Implicit V-Learning

Kaiqiang Ke
Qian Lin
Zongkai Liu
Shenghong He
Chao Yu 0004

Offline goal-conditioned reinforcement learning (GCRL) learns a goal-conditioned value function to train policies for diverse goals with pre-collected datasets. Hindsight experience replay addresses the issue of sparse rewards by treating intermediate states as goals but fails to complete goal-stitching tasks where achieving goals requires stitching different trajectories. While cross-trajectory sampling is a potential solution that associates states and goals belonging to different trajectories, we demonstrate that this direct method degrades performance in goal-conditioned tasks due to the overestimation of values on unconnected pairs. To this end, we propose Conservative Goal-Conditioned Implicit Value Learning (CGCIVL), a novel algorithm that introduces a penalty term to penalize value estimation for unconnected state-goal pairs and leverages the quasimetric framework to accurately estimate values for connected pairs. Evaluations on OGBench, a benchmark for offline GCRL, demonstrate that CGCIVL consistently surpasses state-of-the-art methods across diverse tasks.

Details