Learning to Solve Tasks with Exploring Prior Behaviours

Demonstrations are widely used in Deep Reinforcement Learning (DRL) for facilitating solving tasks with sparse rewards. However, the tasks in real-world scenarios can often have varied initial conditions from the demonstration, which would require additional prior behaviours. For example, consider we are given the demonstration for the task of picking up an object from an open drawer, but the drawer is closed in the training. Without acquiring the prior behaviours of opening the drawer, the robot is unlikely to solve the task. To address this, in this paper we propose an Intrinsic Reward Driven Example-based Control (IRDEC). Our method can endow agents with the ability to explore and acquire the required prior behaviours and then connect to the task-specific behaviours in the demonstration to solve sparse-reward tasks without requiring additional demonstration of the prior behaviours. The performance of our method outperforms other baselines on three navigation tasks and one robotic manipulation task with sparse rewards. Codes are available at https://github.com/Ricky-Zhu/IRDEC.

Authors

• Ruiqi Zhu
• Siyuan Li 0003
• Tianhong Dai
• Chongjie Zhang
• Oya Çeliktutan

Keywords

• Training
• Deep learning
• Codes
• Navigation
• Reinforcement learning
• Task analysis
• Intelligent robots
• Deep Reinforcement Learning
• Robot Manipulator
• Navigation Task
• Robotic Tasks
• Intrinsic Rewards
• Demonstration Of Behavior
• Dynamic Model
• Prediction Error
• State Space
• Future Conditions
• Stationary Distribution
• Forward Model
• State Representation
• Reward Function
• Action Representation
• Markov Decision Process
• Score Estimation
• Update Step
• Target Task
• Observation Space
• Extrinsic Rewards
• State-action Pair
• Ablation Analysis
• Imitation Learning