Object-Category Aware Reinforcement Learning

Object-oriented reinforcement learning (OORL) is a promising way to improve the sample efficiency and generalization ability over standard RL. Recent works that try to solve OORL tasks without additional feature engineering mainly focus on learning the object representations and then solving tasks via reasoning based on these object representations. However, none of these works tries to explicitly model the inherent similarity between different object instances of the same category. Objects of the same category should share similar functionalities; therefore, the category is the most critical property of an object. Following this insight, we propose a novel framework named Object-Category Aware Reinforcement Learning (OCARL), which utilizes the category information of objects to facilitate both perception and reasoning. OCARL consists of three parts: (1) Category-Aware Unsupervised Object Discovery (UOD), which discovers the objects as well as their corresponding categories; (2) Object-Category Aware Perception, which encodes the category information and is also robust to the incompleteness of (1) at the same time; (3) Object-Centric Modular Reasoning, which adopts multiple independent and object-category-specific networks when reasoning based on objects. Our experiments show that OCARL can improve both the sample efficiency and generalization in the OORL domain.

Authors

• Qi Yi University of Science and Technology of China State Key Lab of Processors, Institute of Computing Technology, CAS Cambricon Technologies
• Rui Zhang
• Shaohui Peng Intelligent Software Research Center, Institute of Software, CAS
• Jiaming Guo State Key Lab of Processors, Institute of Computing Technology, CAS
• Xing Hu SKL of Processors, Institute of Computing Technology, CAS
• Zidong Du SKL of Processors, Institute of Computing Technology, CAS
• Xishan Zhang SKL of Processors, Institute of Computing Technology, CAS
• Qi Guo SKL of Processors, Institute of Computing Technology, CAS
• Yunji Chen SKL of Processors, Institute of Computing Technology, CAS University of Chinese Academy of Sciences

Keywords

No keywords are indexed for this paper.