Dynamics Adapted Imitation Learning

We consider Imitation Learning with dynamics variation between the expert demonstration (source domain) and the environment (target domain). Based on the popular framework of Adversarial Imitation Learning, we propose a novel algorithm – Dynamics Adapted Imitation Learning (DYNAIL), which incorporates the dynamics variation into the state-action occupancy measure matching as a regularization term. The dynamics variation is modeled by a pair of classifiers to distinguish between source dynamics and target dynamics. Theoretically, we provide an upper bound on the divergence between the learned policy and expert demonstrations in the source domain. Our error bound only depends on the expectation of the discrepancy between the source and target dynamics for the optimal policy in the target domain. The experiment evaluation validates that our method achieves superior results on high dimensional continuous control tasks, compared to existing imitation learning methods

Authors

• Zixuan Liu
• Liu Liu Tencent AI Lab
• Bingzhe Wu AI Lab, Tencent, Shenzhen, China
• Lanqing Li Tencent
• Xueqian Wang Tsinghua Shenzhen International Graduate School, Tsinghua University
• Bo Yuan University of Queensland
• Peilin Zhao Tencent AI Lab

Keywords

No keywords are indexed for this paper.