DR2: Revisiting Visual Reinforcement Learning from the Dimensional Analysis Perspective

Despite impressive progress on visual control challenges, visual reinforcement learning (VRL) remains sample-inefficient. Existing work commonly leverages auxiliary objectives and data augmentation to learn discriminative representations from observation space containing redundant and task-irrelevant information. However, our analysis shows that the learned representation space still contain dimensional redundancy and dimensional confounders, impeding policy learning. To address these problems, we introduce DR2, a simple plug-and-play module that first constructs a redundancy-reduced representation space and then identifies dimensions most critical for decision making. Concretely, DR2 first applies a redundancy-reduction regularizer to decorrelate latent dimensions, then learns a dimensional mask that models each dimension’s gradient contribution to policy learning, dynamically downweighting task-irrelevant confounders during training. Across diversevisualcontrolbenchmarks, DR2consistentlyimprovessample efficiency and generalization over state-of-the-art baselines. These results indicate that addressing redundancy and confounding at the representation level provides a complementary—rather than substitutive—benefit to existing augmentation and self-supervised strategies.

Authors

• Chuxiong Sun National Key Laboratory of Space Integrated Information System, Institute of Software Chinese Academy of Sciences
• Jinli Chen Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences
• Zehua Zang Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences
• Jiangmeng Li National Key Laboratory of Space Integrated Information System, Institute of Software, Chinese Academy of Sciences
• Rui Wang National Key Laboratory of Space Integrated Information System, Institute of Software Chinese Academy of Sciences
• Changwen Zheng National Key Laboratory of Space Integrated Information System, Institute of Software Chinese Academy of Sciences

Keywords

• Reinforcement Learning
• Visual Reinforcement Learning