Explaining Imitation Learning Through Frames

As one of the prevalent methods to achieve automation systems, imitation learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable AI, we proposed a model-agnostic explaining framework for IL models called Remove and Retrain via Randomized Input Sampling for Explanation (R2RISE). R2RISE aims to explain the importance of frames with respect to the overall policy performance. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames’ importance equality, the effectiveness of the importance map, and connections in importance maps from different IL models. The result shows that R2RISE distinguishes important frames from the demonstrations effectively.

Authors

• Boyuan Zheng
• Jianlong Zhou The Data Science Institute, University of Technology Sydney
• Chunjie Liu
• Yiqiao Li
• Fang Chen School of Biomedical Engineering, and Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

Keywords

• Computational modeling
• Intelligent systems
• Trajectory
• Closed box
• Australia
• Degradation
• Decision making
• Imitation learning
• Neural Network
• Decision-making Process
• Convolutional Neural Network
• Computer Vision
• Single Image
• Generative Adversarial Networks
• Image Frames
• Field Of Computer Vision
• Reinforcement Learning Model
• Explainable Artificial Intelligence
• Wide Range Of Domains
• Model-free Reinforcement Learning
• Inverse Reinforcement Learning
• Community In Recent Years
• Input Trajectory
• Linear Combination
• Image Classification
• Target Model
• Width Of The Image
• Proximal Policy Optimization
• Saliency Map
• Average Return
• Level Of Degradation