Cross-context Visual Imitation Learning from Demonstrations

Imitation learning enables robots to learn a task by simply watching the demonstration of the task. Current imitation learning methods usually require the learner and demonstrator to occur in the same context. This limits their scalability to practical applications. In this paper, we propose a more general imitation learning method which allows the learner and the demonstrator to come from different contexts, such as different viewpoints, backgrounds, and object positions and appearances. Specifically, we design a robotic system consisting of three models: context translation model, depth prediction model and multi-modal inverse dynamics model. First, the context translation model translates the demonstration to the context of learner from a different context. Then combining the color observation and depth observation as inputs, the inverse model maps the multi-modal observations into actions to reproduce the demonstration, where the depth observation is provided by a depth prediction model. By performing the block stacking tasks both in simulation and real world, we prove the cross-context learning advantage of the proposed robotic system over other systems.

Authors

• Shuo Yang
• Wei Zhang 0021
• Weizhi Lu
• Hesheng Wang 0001
• Yibin Li 0001

Keywords

• Context modeling
• Robots
• Task analysis
• Inverse problems
• Visualization
• Predictive models
• Feature extraction
• Visual Learning
• Imitation Learning
• Inverse Reinforcement Learning
• Robotic System
• Inverse Model
• Object Position
• Translational Model
• Multimodal Model
• Supervised Learning
• Paired Data
• Visual Observation
• High Success Rate
• Robotic Arm
• Reward Function
• Real-world Experiments
• General Situation
• Viewpoint Changes
• Robot Learning
• Random Exploration