Author name cluster

Bingyi Kang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

29 papers

2 author rows

ICML Conference 2025 Conference Paper

How Far Is Video Generation from World Model: A Physical Law Perspective

Bingyi Kang
Yang Yue
Rui Lu 0001
Zhijie Lin 0001
Yang Zhao 0003
Kaixin Wang
Gao Huang 0001
Jiashi Feng

Scaling video generation models is believed to be promising in building world models that adhere to fundamental physical laws. However, whether these models can discover physical laws purely from vision can be questioned. A world model learning the true law should give predictions robust to nuances and correctly extrapolate on unseen scenarios. In this work, we evaluate across three key scenarios: in-distribution, out-of-distribution, and combinatorial generalization. We developed a 2D simulation testbed for object movement and collisions to generate videos deterministically governed by one or more classical mechanics laws. We focus on the scaling behavior of training diffusion-based video generation models to predict object movements based on initial frames. Our scaling experiments show perfect generalization within the distribution, measurable scaling behavior for combinatorial generalization, but failure in out-of-distribution scenarios. Further experiments reveal two key insights about the generalization mechanisms of these models: (1) the models fail to abstract general physical rules and instead exhibit "case-based" generalization behavior, i. e. , mimicking the closest training example; (2) when generalizing to new cases, models are observed to prioritize different factors when referencing training data: color $>$ size $>$ velocity $>$ shape. Our study suggests that scaling alone is insufficient for video generation models to uncover fundamental physical laws.

Details

EAAI Journal 2025 Journal Article

Influence factor-based transformation method for translating mass function to probability in Dempster–Shafer evidence theory

Haocheng Shao
Lipeng Pan
Jiahui Chen
Xiaozhuan Gao
Bingyi Kang

Dempster–Shafer evidence theory provides an effective mathematical tool to represent uncertain information by assigning information into power set. Among its associated studies, a pivotal challenge is the transformation of mass functions into probability distributions which can enhance the robustness and reliability of decision-making. In this paper, influence factor is constructed by considering the impact of transformation between multi-element propositions and single-element propositions. Then based on influence factor, the novel transformation method is proposed. In addition, some numerical examples are used to explain effectiveness of new method by analyzing the probability information capacity of different methods. Finally, this paper applies the novel method to target recognition and validates its effectiveness as well as its enhanced support for decision-making through the utilization of real-world datasets.

Details DOI

EAAI Journal 2024 Journal Article

A novel belief Rényi divergence based on belief and plausibility function and its applications in multi-source data fusion

Xiaofei Jin
Yuhang Chang
Huimin Zhang
Bingyi Kang
Jianfeng Zhang

Divergence could be used to quantify the difference between two probability distributions. However, how to accurately quantify the disparity between evidences remains a pending question. To tackle this problem, the present study introduces a novel belief Rényi divergence for more accurately quantifying the difference between evidences. The devised divergence incorporates not only the median value of belief interval of the mass function but also considers the impact of the interval range. It enables a comprehensive reflection of the relationships among different subsets. Furthermore, the proposed divergence measure is shown to be connected to the Hellinger distance, Kullback–Leibler divergence, and χ 2 divergence as α varies. A set of properties is then derived, expanding the range of potential applications for this divergence. Subsequently, a novel multi-source data fusion approach built upon the introduced divergence metric is developed. Finally, the soundness and superiority of the approach are displayed through experimental validation and comprehensive analysis.

Details DOI

NeurIPS Conference 2024 Conference Paper

Classification Done Right for Vision-Language Pre-Training

Zilong Huang
Qinghao Ye
Bingyi Kang
Jiashi Feng
Haoqi Fan

We introduce SuperClass, a super simple classification method for vision-language pre-training on image-text data. Unlike its contrastive counterpart CLIP who contrast with a text encoder, SuperClass directly utilizes tokenized raw text as supervised classification labels, without the need for additional text filtering or selection. Due to the absence of the text encoding as contrastive target, SuperClass does not require a text encoder and does not need to maintain a large batch size as CLIP does. SuperClass demonstrated superior performance on various downstream tasks, including classic computer vision benchmarks and vision language downstream tasks. We further explored the scaling behavior of SuperClass on model size, training length, or data size, and reported encouraging results and comparisons to CLIP. https: //github. com/x-cls/superclass

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution

Yang Yue
Yulin Wang
Bingyi Kang
Yizeng Han
Shenzhi Wang
Shiji Song
Jiashi Feng
Gao Huang

Multimodal Large Language Models (MLLMs) have demonstrated remarkable comprehension and reasoning capabilities with complex language and visual data. These advances have spurred the vision of establishing a generalist robotic MLLM proficient in understanding complex human instructions and accomplishing various embodied tasks, whose feasibility has been recently verified~\cite{rt-2, rt-x}. However, developing MLLMs for real-world robots is challenging due to the typically limited computation and memory capacities available on robotic platforms. In contrast, the inference of MLLMs usually incorporates storing billions of parameters and performing tremendous computation, imposing significant hardware demands. In our paper, we seek to address this challenge by leveraging an intriguing observation: relatively easier situations make up the bulk of the procedure of controlling robots to fulfill diverse tasks, and they generally require far smaller models to obtain the correct robotic actions. Motivated by this observation, we propose a \emph{DynamicEarly-Exit for Robotic MLLM} (DeeR) framework that automatically adjusts the size of the activated MLLM based on each situation at hand. The approach leverages a multi-exit architecture in MLLMs, which allows the model to cease processing once a proper size of the model has been activated for a specific situation, thus avoiding further redundant computation. Additionally, we develop novel algorithms that establish early-termination criteria for DeeR, conditioned on predefined demands such as average computational cost (\emph{i. e. }, power consumption), as well as peak computational consumption (\emph{i. e. }, latency) and GPU memory usage. These enhancements ensure that DeeR operates efficiently under varying resource constraints while maintaining competitive performance. Moreover, we design a tailored training method for integrating temporal information on top of such multi-exit architectures to predict actions reasonably. On the CALVIN robot manipulation benchmark, DeeR demonstrates significant reductions in computational costs by 5. 2-6. 5x and GPU memory by 2x without compromising performance. Code and checkpoints are available at https: //github. com/yueyang130/DeeR-VLA.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Depth Anything V2

Lihe Yang
Bingyi Kang
Zilong Huang
Zhen Zhao
Xiaogang Xu
Jiashi Feng
Hengshuang Zhao

This work presents Depth Anything V2. Without pursuing fancy techniques, we aim to reveal crucial findings to pave the way towards building a powerful monocular depth estimation model. Notably, compared with V1, this version produces much finer and more robust depth predictions through three key practices: 1) replacing all labeled real images with synthetic images, 2) scaling up the capacity of our teacher model, and 3) teaching student models via the bridge of large-scale pseudo-labeled real images. Compared with the latest models built on Stable Diffusion, our models are significantly more efficient (more than 10x faster) and more accurate. We offer models of different scales (ranging from 25M to 1. 3B params) to support extensive scenarios. Benefiting from their strong generalization capability, we fine-tune them with metric depth labels to obtain our metric depth models. In addition to our models, considering the limited diversity and frequent noise in current test sets, we construct a versatile evaluation benchmark with sparse depth annotations to facilitate future research. Models are available at https: //github. com/DepthAnything/Depth-Anything-V2.

PDF Details DOI

EAAI Journal 2024 Journal Article

Evaluating the reliability and relative weight of the evidence using approximate evidential mutual information

Xuan Zhao
Mengyi Zhang
Zilong Xiao
Bingyi Kang

How to evaluate the reliability and relative weight of the evidence play a key role in improving the performance of the model using Dempster–Shafer(D–S) evidence theory. A new method of evaluating the reliability and relative weight of the evidence using approximate evidential mutual information is proposed. First, the reliability of evidence sources is evaluated based on the approximate mutual information between evidence sets as the global reliability. Second, the relative weight of other evidence is obtained according to the similarity relative to the optimal global reliability. Third, the comprehensive quality of each evidence is measured by aggregating the global reliability and relative weight as a discount factor. Finally, all corrected evidence is fused by the Dempster combination rule to obtain a final result. The proposed method has better performance and strong stability in the evaluation of plant taxonomy based on morphological characteristics, which provides a reliable basis for plant taxonomy.

Details DOI

NeurIPS Conference 2024 Conference Paper

Image Understanding Makes for A Good Tokenizer for Image Generation

Luting Wang
Yang Zhao
Zijian Zhang
Jiashi Feng
Si Liu
Bingyi Kang

Modern image generation (IG) models have been shown to capture rich semantics valuable for image understanding (IU) tasks. However, the potential of IU models to improve IG performance remains uncharted. We address this issue using a token-based IG framework, which relies on effective tokenizers to project images into token sequences. Currently, **pixel reconstruction** (e. g. , VQGAN) dominates the training objective for image tokenizers. In contrast, our approach adopts the **feature reconstruction** objective, where tokenizers are trained by distilling knowledge from pretrained IU encoders. Comprehensive comparisons indicate that tokenizers with strong IU capabilities achieve superior IG performance across a variety of metrics, datasets, tasks, and proposal networks. Notably, VQ-KD CLIP achieves $4. 10$ FID on ImageNet-1k (IN-1k). Visualization suggests that the superiority of VQ-KD can be partly attributed to the rich semantics within the VQ-KD codebook. We further introduce a straightforward pipeline to directly transform IU encoders into tokenizers, demonstrating exceptional effectiveness for IG tasks. These discoveries may energize further exploration into image tokenizer research and inspire the community to reassess the relationship between IU and IG. The code is released at https: //github. com/magic-research/vector_quantization.

PDF Details DOI

ICML Conference 2024 Conference Paper

Improving Token-Based World Models with Parallel Observation Prediction

Lior Cohen
Kaixin Wang
Bingyi Kang
Shie Mannor

Motivated by the success of Transformers when applied to sequences of discrete symbols, token-based world models (TBWMs) were recently proposed as sample-efficient methods. In TBWMs, the world model consumes agent experience as a language-like sequence of tokens, where each observation constitutes a sub-sequence. However, during imagination, the sequential token-by-token generation of next observations results in a severe bottleneck, leading to long training times, poor GPU utilization, and limited representations. To resolve this bottleneck, we devise a novel Parallel Observation Prediction (POP) mechanism. POP augments a Retentive Network (RetNet) with a novel forward mode tailored to our reinforcement learning setting. We incorporate POP in a novel TBWM agent named REM (Retentive Environment Model), showcasing a 15. 4x faster imagination compared to prior TBWMs. REM attains superhuman performance on 12 out of 26 games of the Atari 100K benchmark, while training in less than 12 hours. Our code is available at https: //github. com/leor-c/REM

Details

NeurIPS Conference 2024 Conference Paper

MADiff: Offline Multi-agent Learning with Diffusion Models

Zhengbang Zhu
Minghuan Liu
Liyuan Mao
Bingyi Kang
Minkai Xu
Yong Yu
Stefano Ermon
Weinan Zhang

Offline reinforcement learning (RL) aims to learn policies from pre-existing datasets without further interactions, making it a challenging task. Q-learning algorithms struggle with extrapolation errors in offline settings, while supervised learning methods are constrained by model expressiveness. Recently, diffusion models (DMs) have shown promise in overcoming these limitations in single-agent learning, but their application in multi-agent scenarios remains unclear. Generating trajectories for each agent with independent DMs may impede coordination, while concatenating all agents’ information can lead to low sample efficiency. Accordingly, we propose MADiff, which is realized with an attention-based diffusion model to model the complex coordination among behaviors of multiple agents. To our knowledge, MADiff is the first diffusion-based multi-agent learning framework, functioning as both a decentralized policy and a centralized controller. During decentralized executions, MADiff simultaneously performs teammate modeling, and the centralized controller can also be applied in multi-agent trajectory predictions. Our experiments demonstrate that MADiff outperforms baseline algorithms across various multi-agent learning tasks, highlighting its effectiveness in modeling complex multi-agent interactions.

PDF Details DOI

ICML Conference 2023 Conference Paper

Bag of Tricks for Training Data Extraction from Language Models

Weichen Yu
Tianyu Pang
Qian Liu 0033
Chao Du
Bingyi Kang
Yan Huang
Min Lin
Shuicheng Yan

With the advance of language models, privacy protection is receiving more attention. Training data extraction is therefore of great importance, as it can serve as a potential tool to assess privacy leakage. However, due to the difficulty of this task, most of the existing methods are proof-of-concept and still not effective enough. In this paper, we investigate and benchmark tricks for improving training data extraction using a publicly available dataset. Because most existing extraction methods use a pipeline of generating-then-ranking, i. e. , generating text candidates as potential training data and then ranking them based on specific criteria, our research focuses on the tricks for both text generation (e. g. , sampling strategy) and text ranking (e. g. , token-level criteria). The experimental results show that several previously overlooked tricks can be crucial to the success of training data extraction. Based on the GPT-Neo 1. 3B evaluation results, our proposed tricks outperform the baseline by a large margin in most cases, providing a much stronger baseline for future research. The code is available at https: //github. com/weichen-yu/LM-Extraction.

Details

EAAI Journal 2023 Journal Article

BGC: Belief gravitational clustering approach and its application in the counter-deception of belief functions

Huizi Cui
Huaqing Zhang
Yuhang Chang
Bingyi Kang

Counter-deception information fusion is a significant issue in Dempster–Shafer evidence theory (DST). How to effectively counter the deception is the key problem in belief function. Limited work has been presented, of which the negation view, degree of falsity view, and the minimum conflict view are popular ones. However, previous work may suffer from combinatorial explosion or be limited to simple practical application. Based on our previous belief universal gravitation (BUG) model, a simple belief gravitational clustering (BGC) is proposed to model the evidential clustering process. Some goals, like no parameter adjustment, no initial condition, objective and unique cluster number, and robustness are achieved. Furthermore, BGC-based fusion strategy is ulteriorly raised to determine whether the evidence should be fused, which fully considers the abnormal feature of the body of evidence and the nature of the combination rule to perceive the deception. Both theoretical analysis and experimental results demonstrate the accuracy and flexibility of the proposed work.

Details DOI

NeurIPS Conference 2023 Conference Paper

Efficient Diffusion Policies For Offline Reinforcement Learning

Bingyi Kang
Xiao Ma
Chao Du
Tianyu Pang
Shuicheng Yan

Offline reinforcement learning (RL) aims to learn optimal policies from offline datasets, where the parameterization of policies is crucial but often overlooked. Recently, Diffsuion-QL significantly boosts the performance of offline RL by representing a policy with a diffusion model, whose success relies on a parametrized Markov Chain with hundreds of steps for sampling. However, Diffusion-QL suffers from two critical limitations. 1) It is computationally inefficient to forward and backward through the whole Markov chain during training. 2) It is incompatible with maximum likelihood-based RL algorithms (e. g. , policy gradient methods) as the likelihood of diffusion models is intractable. Therefore, we propose efficient diffusion policy (EDP) to overcome these two challenges. EDP approximately constructs actions from corrupted ones at training to avoid running the sampling chain. We conduct extensive experiments on the D4RL benchmark. The results show that EDP can reduce the diffusion policy training time from 5 days to 5 hours on gym-locomotion tasks. Moreover, we show that EDP is compatible with various offline RL algorithms (TD3, CRR, and IQL) and achieves new state-of-the-art on D4RL by large margins over previous methods.

PDF Details

NeurIPS Conference 2023 Conference Paper

FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models

Lihe Yang
Xiaogang Xu
Bingyi Kang
Yinghuan Shi
Hengshuang Zhao

Semantic segmentation has witnessed tremendous progress due to the proposal of various advanced network architectures. However, they are extremely hungry for delicate annotations to train, and the acquisition is laborious and unaffordable. Therefore, we present FreeMask in this work, which resorts to synthetic images from generative models to ease the burden of both data collection and annotation procedures. Concretely, we first synthesize abundant training images conditioned on the semantic masks provided by realistic datasets. This yields extra well-aligned image-mask training pairs for semantic segmentation models. We surprisingly observe that, solely trained with synthetic images, we already achieve comparable performance with real ones (e. g. , 48. 3 vs. 48. 5 mIoU on ADE20K, and 49. 3 vs. 50. 5 on COCO-Stuff). Then, we investigate the role of synthetic images by joint training with real images, or pre-training for real images. Meantime, we design a robust filtering principle to suppress incorrectly synthesized regions. In addition, we propose to inequally treat different semantic masks to prioritize those harder ones and sample more corresponding synthetic images for them. As a result, either jointly trained or pre-trained with our filtered and re-sampled synthesized images, segmentation models can be greatly enhanced, e. g. , from 48. 7 to 52. 0 on ADE20K.

PDF Details

NeurIPS Conference 2023 Conference Paper

Mutual Information Regularized Offline Reinforcement Learning

Xiao Ma
Bingyi Kang
Zhongwen Xu
Min Lin
Shuicheng Yan

The major challenge of offline RL is the distribution shift that appears when out-of-distribution actions are queried, which makes the policy improvement direction biased by extrapolation errors. Most existing methods address this problem by penalizing the policy or value for deviating from the behavior policy during policy improvement or evaluation. In this work, we propose a novel MISA framework to approach offline RL from the perspective of Mutual Information between States and Actions in the dataset by directly constraining the policy improvement direction. MISA constructs lower bounds of mutual information parameterized by the policy and Q-values. We show that optimizing this lower bound is equivalent to maximizing the likelihood of a one-step improved policy on the offline dataset. Hence, we constrain the policy improvement direction to lie in the data manifold. The resulting algorithm simultaneously augments the policy evaluation and improvement by adding mutual information regularizations. MISA is a general framework that unifies conservative Q-learning (CQL) and behavior regularization methods (e. g. , TD3+BC) as special cases. We introduce 3 different variants of MISA, and empirically demonstrate that tighter mutual information lower bound gives better offline RL performance. In addition, our extensive experiments show MISA significantly outperforms a wide range of baselines on various tasks of the D4RL benchmark, e. g. , achieving 742. 9 total points on gym-locomotion tasks. Our code is attached and will be released upon publication.

PDF Details

ICLR Conference 2023 Conference Paper

Revisiting Intrinsic Reward for Exploration in Procedurally Generated Environments

Kaixin Wang
Kuangqi Zhou
Bingyi Kang
Jiashi Feng
Shuicheng Yan

Exploration under sparse rewards remains a key challenge in deep reinforcement learning. Recently, studying exploration in procedurally-generated environments has drawn increasing attention. Existing works generally combine lifelong intrinsic rewards and episodic intrinsic rewards to encourage exploration. Though various lifelong and episodic intrinsic rewards have been proposed, the individual contributions of the two kinds of intrinsic rewards to improving exploration are barely investigated. To bridge this gap, we disentangle these two parts and conduct ablative experiments. We consider lifelong and episodic intrinsic rewards used in prior works, and compare the performance of all lifelong-episodic combinations on the commonly used MiniGrid benchmark. Experimental results show that only using episodic intrinsic rewards can match or surpass prior state-of-the-art methods. On the other hand, only using lifelong intrinsic rewards hardly makes progress in exploration. This demonstrates that episodic intrinsic reward is more crucial than lifelong one in boosting exploration. Moreover, we find through experimental analysis that the lifelong intrinsic reward does not accurately reflect the novelty of states, which explains why it does not help much in improving exploration.

Details

NeurIPS Conference 2023 Conference Paper

Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL

Yang Yue
Rui Lu
Bingyi Kang
Shiji Song
Gao Huang

The divergence of the Q-value estimation has been a prominent issue offline reinforcement learning (offline RL), where the agent has no access to real dynamics. Traditional beliefs attribute this instability to querying out-of-distribution actions when bootstrapping value targets. Though this issue can be alleviated with policy constraints or conservative Q estimation, a theoretical understanding of the underlying mechanism causing the divergence has been absent. In this work, we aim to thoroughly comprehend this mechanism and attain an improved solution. We first identify a fundamental pattern, \emph{self-excitation}, as the primary cause of Q-value estimation divergence in offline RL. Then, we propose a novel Self-Excite Eigenvalue Measure (SEEM) metric based on Neural Tangent Kernel (NTK) to measure the evolving property of Q-network at training, which provides an intriguing explanation of the emergence of divergence. For the first time, our theory can reliably decide whether the training will diverge at an early stage, and even predict the order of the growth for the estimated Q-value, the model's norm, and the crashing step when an SGD optimizer is used. The experiments demonstrate perfect alignment with this theoretic analysis. Building on our insights, we propose to resolve divergence from a novel perspective, namely improving the model's architecture for better extrapolating behavior. Through extensive empirical studies, we identify LayerNorm as a good solution to effectively avoid divergence without introducing detrimental bias, leading to superior performance. Experimental results prove that it can still work in some most challenging settings, i. e. using only 1$\%$ transitions of the dataset, where all previous methods fail. Moreover, it can be easily plugged into modern offline RL methods and achieve SOTA results on many challenging tasks. We also give unique insights into its effectiveness.

PDF Details

AAAI Conference 2023 Conference Paper

Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning

Yang Yue
Bingyi Kang
Zhongwen Xu
Gao Huang
Shuicheng Yan

Deep reinforcement learning (RL) algorithms suffer severe performance degradation when the interaction data is scarce, which limits their real-world application. Recently, visual representation learning has been shown to be effective and promising for boosting sample efficiency in RL. These methods usually rely on contrastive learning and data augmentation to train a transition model, which is different from how the model is used in RL---performing value-based planning. Accordingly, the learned representation by these visual methods may be good for recognition but not optimal for estimating state value and solving the decision problem. To address this issue, we propose a novel method, called value-consistent representation learning (VCR), to learn representations that are directly related to decision-making. More specifically, VCR trains a model to predict the future state (also referred to as the "imagined state'') based on the current one and a sequence of actions. Instead of aligning this imagined state with a real state returned by the environment, VCR applies a Q value head on both of the states and obtains two distributions of action values. Then a distance is computed and minimized to force the imagined state to produce a similar action value prediction as that by the real state. We develop two implementations of the above idea for the discrete and continuous action spaces respectively. We conduct experiments on Atari 100k and DeepMind Control Suite benchmarks to validate their effectiveness for improving sample efficiency. It has been demonstrated that our methods achieve new state-of-the-art performance for search-free RL algorithms.

PDF Details DOI

EAAI Journal 2022 Journal Article

An intelligent quality-based fusion method for complex-valued distributions using POWA operator

Yanan Li
Ruonan Zhu
Xiangjun Mi
Bingyi Kang

Information quality (IQ) is a significant index in information processing, e. g. multiple information fusion. An extended method considering the IQ has been investigated using the multi-complex-valued information. However, how to fuse the multiple complex-valued distributions (CvDs) with correlation and preference of decision-maker may be a fantastic issue. In this paper, we propose a new method using the power ordered weighted average (POWA) operator to integrate multi-complex-valued distributions considering the information correlation and preference of decision-maker. The proposed method is an extension of the previous work. Some examples and applications are used to illustrate the effectiveness of the proposed method.

Details DOI

ICLR Conference 2021 Conference Paper

Exploring Balanced Feature Spaces for Representation Learning

Bingyi Kang
Yu Li 0016
Sa Xie
Zehuan Yuan
Jiashi Feng

Existing self-supervised learning (SSL) methods are mostly applied for training representation models from artificially balanced datasets (e.g., ImageNet). It is unclear how well they will perform in the practical scenarios where datasets are often imbalanced w.r.t. the classes. Motivated by this question, we conduct a series of studies on the performance of self-supervised contrastive learning and supervised learning methods over multiple datasets where training instance distributions vary from a balanced one to a long-tailed one. Our findings are quite intriguing. Different from supervised methods with large performance drop, the self-supervised contrastive learning methods perform stably well even when the datasets are heavily imbalanced. This motivates us to explore the balanced feature spaces learned by contrastive learning, where the feature representations present similar linear separability w.r.t. all the classes. Our further experiments reveal that a representation model generating a balanced feature space can generalize better than that yielding an imbalanced one across multiple settings. Inspired by these insights, we develop a novel representation learning method, called $k$-positive contrastive learning. It effectively combines strengths of the supervised method and the contrastive learning method to learn representations that are both discriminative and balanced. Extensive experiments demonstrate its superiority on multiple recognition tasks. Remarkably, it achieves new state-of-the-art on challenging long-tailed recognition benchmarks. Code and models will be released.

Details

ICLR Conference 2021 Conference Paper

Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control

Zhuang Liu 0003
Xuanlin Li
Bingyi Kang
Trevor Darrell

Deep Reinforcement Learning (Deep RL) has been receiving increasingly more attention thanks to its encouraging performance on a variety of control tasks. Yet, conventional regularization techniques in training neural networks (e.g., $L_2$ regularization, dropout) have been largely ignored in RL methods, possibly because agents are typically trained and evaluated in the same environment, and because the deep RL community focuses more on high-level algorithm designs. In this work, we present the first comprehensive study of regularization techniques with multiple policy optimization algorithms on continuous control tasks. Interestingly, we find conventional regularization techniques on the policy networks can often bring large improvement, especially on harder tasks. Our findings are shown to be robust against training hyperparameter variations. We also compare these techniques with the more widely used entropy regularization. In addition, we study regularizing different components and find that only regularizing the policy network is typically the best. We further analyze why regularization may help generalization in RL from four perspectives - sample complexity, reward distribution, weight norm, and noise robustness. We hope our study provides guidance for future practices in regularizing policy optimization algorithms. Our code is available at https://github.com/xuanlinli17/iclr2021_rlreg .

Details

EAAI Journal 2021 Journal Article

Strategies of attack–defense game for wireless sensor networks considering the effect of confidence level in fuzzy environment

Yingfu Wu
Bingyi Kang
Hao Wu

It is a common case that Wireless Sensor Networks are attacked by malware in the real world. According to the game theory, the action of attack–defense between Wireless Sensor Network(WSN) and malware can be regarded as a game. While substantial efforts have been made to address this issue, most of these efforts have predominantly focused on the analysis of attack–defense game in the known environment. Given that the process of gaming in real world often contains a lot of fuzzy information, we extend the focus in this line by considering the fuzzy exterior environment. Specifically, we assume the WSN attack–defense Stackelberg game is in the fuzzy environment by using fuzzy variable. Then Stackelberg game theory is utilized to calculate the equilibrium solutions of the introduced m a x i m a x chance-constrained model and m i n i m a x chance-constrained model. Based on the simulation data, this study demonstrates the confidence levels and decision perspectives affect the optimal strategy of WSN and the reliability of WSN. Finally, the novel analytical method is compared with the non-fuzzy WSN attack–defense game method. The analysis shows that the novel approach is optimal in terms of predicting the behavior of malware in resisting the attack of malware.

Details DOI

EAAI Journal 2021 Journal Article

Z E -numbers: A new extended Z-numbers and its application on multiple attribute group decision making

Ye Tian
Xiangjun Mi
Yunpeng Ji
Bingyi Kang

As the core mechanism of intelligent systems, decision-making has received widespread attention in recent years. As decision-making environments become more complex, large amounts of data are fuzzy and partially reliable. Zadeh proposed the concept of the Z-numbers, this more anthropomorphic fuzzy set representation framework describes the simultaneous existence of probability measures and probability measures of random variables, and it is regarded as a very powerful tool for modeling uncertain information. However, the representation of Z-numbers still has limitations. Therefore, we propose a new extended Z-numbers, Z E = ( ( A, B ), E ), E is the credibility. As the objective reliability of ( A, B ), it restricts the original Z-numbers. At the same time, the conversion function between them is also defined. Based on this, we proposed a multi-attribute group decision-making (MAGDM) method considering the attitudes of decision-makers. Application examples show the rationality and effectiveness of the proposed methodology, and the superiority of this method is further illustrated through comparison and discussion with other methods.

Details DOI

ICLR Conference 2020 Conference Paper

Decoupling Representation and Classifier for Long-Tailed Recognition

Bingyi Kang
Saining Xie
Marcus Rohrbach
Zhicheng Yan 0001
Albert Gordo
Jiashi Feng
Yannis Kalantidis

The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but most of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification. Our code is available at https://github.com/facebookresearch/classifier-balancing.

Details

NeurIPS Conference 2020 Conference Paper

Improving Generalization in Reinforcement Learning with Mixture Regularization

Kaixin Wang
Bingyi Kang
Jie Shao
Jiashi Feng

Deep reinforcement learning (RL) agents trained in a limited set of environments tend to suffer overfitting and fail to generalize to unseen testing environments. To improve their generalizability, data augmentation approaches (e. g. cutout and random convolution) are previously explored to increase the data diversity. However, we find these approaches only locally perturb the observations regardless of the training environments, showing limited effectiveness on enhancing the data diversity and the generalization performance. In this work, we introduce a simple approach, named mixreg, which trains agents on a mixture of observations from different training environments and imposes linearity constraints on the observation interpolations and the supervision (e. g. associated reward) interpolations. Mixreg increases the data diversity more effectively and helps learn smoother policies. We verify its effectiveness on improving generalization by conducting extensive experiments on the large-scale Procgen benchmark. Results show mixreg outperforms the well-established baselines on unseen testing environments by a large margin. Mixreg is simple, effective and general. It can be applied to both policy-based and value-based RL algorithms. Code is available at https: //github. com/kaixin96/mixreg.

PDF Details

EAAI Journal 2019 Journal Article

Derive knowledge of Z-number from the perspective of Dempster–Shafer evidence theory

Qing Liu
Ye Tian
Bingyi Kang

Z-number, combined with constraint and reliability of the information, is an effective frame to simulate the thinking of humans. How to derive knowledge of Z-numbers, especially from the objective data may become a fascinating and open issue. In this paper, a method of deriving knowledge of Z-numbers from the perspective of Dempster–Shafer theory is proposed. The proposed method considers the Z-number generating from objective and subjective data using Dempster–Shafer theory. Some numerical examples and experimental simulations are used to illustrate the effectiveness of the proposed methodology.

Details DOI

ICML Conference 2018 Conference Paper

Policy Optimization with Demonstrations

Bingyi Kang
Zequn Jie
Jiashi Feng

Exploration remains a significant challenge to reinforcement learning methods, especially in environments where reward signals are sparse. Recent methods of learning from demonstrations have shown to be promising in overcoming exploration difficulties but typically require considerable high-quality demonstrations that are difficult to collect. We propose to effectively leverage available demonstrations to guide exploration through enforcing occupancy measure matching between the learned policy and current demonstrations, and develop a novel Policy Optimization from Demonstration (POfD) method. We show that POfD induces implicit dynamic reward shaping and brings provable benefits for policy improvement. Furthermore, it can be combined with policy gradient methods to produce state-of-the-art results, as demonstrated experimentally on a range of popular benchmark sparse-reward tasks, even when the demonstrations are few and imperfect.

Details

IJCAI Conference 2018 Conference Paper

Sharing Residual Units Through Collective Tensor Factorization To Improve Deep Neural Networks

Yunpeng Chen
Xiaojie Jin
Bingyi Kang
Jiashi Feng
Shuicheng Yan

The residual unit and its variations are wildly used in building very deep neural networks for alleviating optimization difficulty. In this work, we revisit the standard residual function as well as its several successful variants and propose a unified framework based on tensor Block Term Decomposition (BTD) to explain these apparently different residual functions from the tensor decomposition view. With the BTD framework, we further propose a novel basic network architecture, named the Collective Residual Unit (CRU). CRU further enhances parameter efficiency of deep residual neural networks by sharing core factors derived from collective tensor factorization over the involved residual units. It enables efficient knowledge sharing across multiple residual units, reduces the number of model parameters, lowers the risk of over-fitting, and provides better generalization ability. Extensive experimental results show that our proposed CRU network brings outstanding parameter efficiency -- it achieves comparable classification performance with ResNet-200 while using a model size as small as ResNet-50 on the ImageNet-1k and Places365-Standard benchmark datasets.

PDF Details

UAI Conference 2018 Conference Paper

Transferable Meta Learning Across Domains

Bingyi Kang
Jiashi Feng

Meta learning algorithms are effective at obtaining meta models with the capability of solving new tasks quickly. However, they critically require sufﬁcient tasks for meta model training and the resulted model can only solve new tasks similar to the training ones. These limitations make them suffer performance decline in presence of insufﬁciency of training tasks in target domains and task heterogeneity—the source (model training) tasks presents different characteristics from target (model application) tasks. To overcome these two significant limitations of existing meta learning algorithms, we introduce the cross-domain meta learning framework and propose a new transferable meta learning (TML) algorithm. TML performs meta task adaptation jointly with meta model learning, which effectively narrows divergence between source and target tasks and enables transferring source metaknowledge to solve target tasks. Thus, the resulted transferable meta model can solve new learning tasks in new domains quickly. We apply the proposed TML to cross-domain fewshot classiﬁcation problems and evaluate its performance on multiple benchmarks. It performs signiﬁcantly better and faster than wellestablished meta learning algorithms and ﬁnetuned domain-adapted models.

Details