Author name cluster

Shihab Ahmed

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

AAMAS Conference 2026 Conference Paper

Robust Direct Preference Optimization for Offline Learning

Shihab Ahmed
Zhenyi Wang
Yue Wang

Recent preference-based alignment methods, such as Direct Preference Optimization (DPO), perform well under comprehensive offline datasets. However, practical datasets often contain sparse, noisy, and unevenly distributed comparisons, which can degrade model performance. To address this, we first adopt the principle of pessimism and propose a Robust DPO framework that optimizes for the worst-case reward within some data-dependent uncertainty set, and show its effectiveness in offline problems. Moreover, we show that the resulting robust optimal policy can be obtained by directly fine-tuning a baseline DPO model, avoiding the need for retraining. We further construct an uncertainty set to tackle the data uncertainty, based on the graph Laplacian, and show the set contains the true underlying reward with high probability. We then further evaluate the effectiveness of our method in controlled tabular and LLM setting, which validate our theoretical finds.

PDF

AAAI Conference 2026 Conference Paper

Stabilizing Policy Gradient Methods via Reward Profiling

Shihab Ahmed
El Houcine Bergou
Yue Wang
Aritra Dutta

Policy gradient methods, which have been extensively studied in the last decade, offer an effective and efficient framework for reinforcement learning problems. However, their performances can often be unsatisfactory, suffering from unreliable reward improvements and slow convergence, due to high variance in gradient estimations. In this paper, we propose a universal reward profiling framework that can be seamlessly integrated with any policy gradient algorithm, where we selectively update the policy based on high-confidence performance estimations. We theoretically justify that our technique will not slow down the convergence of the baseline policy gradient methods, but with high probability, will result in stable and monotonic improvements of their performance. Empirically, on eight continuous‐control benchmarks (Box2D and MuJoCo/PyBullet), our profiling yields up to 1.5x faster convergence to near‐optimal returns, up to 1.75x reduction in return variance on some setups. Our profiling approach offers a general, theoretically grounded path to more reliable and efficient policy learning in complex environments.

PDF Details DOI