Author name cluster

Ling Shao 0001

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

1 author row

ICLR Conference 2022 Conference Paper

Hierarchical Variational Memory for Few-shot Learning Across Domains

Yingjun Du
Xiantong Zhen
Ling Shao 0001
Cees G. M. Snoek

Neural memory enables fast adaptation to new tasks with just a few training samples. Existing memory models store features only from the single last layer, which does not generalize well in presence of a domain shift between training and test distributions. Rather than relying on a flat memory, we propose a hierarchical alternative that stores features at different semantic levels. We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We meta-learn the model by a newly derived hierarchical variational inference framework, where hierarchical memory and prototypes are jointly optimized. To explore and exploit the importance of different semantic levels, we further propose to learn the weights associated with the prototype at each level in a data-driven way, which enables the model to adaptively choose the most generalizable features. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model. The new state-of-the-art performance on cross-domain and competitive performance on traditional few-shot classification further substantiates the benefit of hierarchical variational memory.

Details

ICLR Conference 2022 Conference Paper

Learning to Generalize across Domains on Single Test Samples

Zehao Xiao
Xiantong Zhen
Ling Shao 0001
Cees G. M. Snoek

We strive to learn a model from a set of source domains that generalizes well to unseen target domains. The main challenge in such a domain generalization scenario is the unavailability of any target domain data during training, resulting in the learned model not being explicitly adapted to the unseen target domains. We propose learning to generalize across domains on single test samples. We leverage a meta-learning paradigm to learn our model to acquire the ability of adaptation with single samples at training time so as to further adapt itself to each single test sample at test time. We formulate the adaptation to the single test sample as a variational Bayesian inference problem, which incorporates the test sample as a conditional into the generation of model parameters. The adaptation to each test sample requires only one feed-forward computation at test time without any fine-tuning or self-supervised training on additional data from the unseen domains. Extensive ablation studies demonstrate that our model learns the ability to adapt models to each single sample by mimicking domain shifts during training. Further, our model achieves at least comparable -- and often better -- performance than state-of-the-art methods on multiple benchmarks for domain generalization.

Details

ICML Conference 2021 Conference Paper

A Bit More Bayesian: Domain-Invariant Learning with Uncertainty

Zehao Xiao
Jiayi Shen
Xiantong Zhen
Ling Shao 0001
Cees G. M. Snoek

Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.

Details

ICML Conference 2021 Conference Paper

Kernel Continual Learning

Mohammad Mahdi Derakhshani
Xiantong Zhen
Ling Shao 0001
Cees G. M. Snoek

This paper introduces kernel continual learning, a simple but effective variant of continual learning that leverages the non-parametric nature of kernel methods to tackle catastrophic forgetting. We deploy an episodic memory unit that stores a subset of samples for each task to learn task-specific classifiers based on kernel ridge regression. This does not require memory replay and systematically avoids task interference in the classifiers. We further introduce variational random features to learn a data-driven kernel for each task. To do so, we formulate kernel continual learning as a variational inference problem, where a random Fourier basis is incorporated as the latent variable. The variational posterior distribution over the random Fourier basis is inferred from the coreset of each task. In this way, we are able to generate more informative kernels specific to each task, and, more importantly, the coreset size can be reduced to achieve more compact memory, resulting in more efficient continual learning based on episodic memory. Extensive evaluation on four benchmarks demonstrates the effectiveness and promise of kernels for continual learning.

Details

ICLR Conference 2021 Conference Paper

MetaNorm: Learning to Normalize Few-Shot Batches Across Domains

Yingjun Du
Xiantong Zhen
Ling Shao 0001
Cees G. M. Snoek

Batch normalization plays a crucial role when training deep neural networks. However, batch statistics become unstable with small batch sizes and are unreliable in the presence of distribution shifts. We propose MetaNorm, a simple yet effective meta-learning normalization. It tackles the aforementioned issues in a unified way by leveraging the meta-learning setting and learns to infer adaptive statistics for batch normalization. MetaNorm is generic, flexible and model-agnostic, making it a simple plug-and-play module that is seamlessly embedded into existing meta-learning approaches. It can be efficiently implemented by lightweight hypernetworks with low computational cost. We verify its effectiveness by extensive evaluation on representative tasks suffering from the small batch and domain shift problems: few-shot learning and domain generalization. We further introduce an even more challenging setting: few-shot domain generalization. Results demonstrate that MetaNorm consistently achieves better, or at least competitive, accuracy compared to existing batch normalization methods.

Details

ICRA Conference 2021 Conference Paper

Target-targeted Domain Adaptation for Unsupervised Semantic Segmentation

Xiaohong Zhang 0009
Haofeng Zhang 0001
Jianfeng Lu 0003
Ling Shao 0001
Jingyu Yang 0001

Semantic segmentation has attracted increasing attention due to its important role in self-driving, and it is often realized by supervised learning with large number of well labeled maps. However, the labeled images are hard to be obtained in most circumstances, and the common way for unsupervised semantic segmentation is usually implemented by transferring the knowledge from source supervised domain to target unsupervised domain. Most researches focus on encouraging target predictions to be closer to the source ones through a weight-sharing network, and achieve certain performance. However, these methods often suffer from the domain shift problem that the networks are often trained towards the source domain and lead to performance degradation. In this paper, we propose a target-targeted domain adaptation approach by focusing the training on target domain. Our model consists of two components: the Image-to-image Translation (IIT) module to translate the source image to target domain and the Target-targeted Segmentation Adaptation (TSA) module to focus the semantic segmentation on target domain. The IIT module deals with image space alignment while the TSA module bridges the domain gap at the segmentation map level. In addition, we design a closed-loop learning to promote each other by employing feedback from TSA to IIT. Extensive experiments on GTA5 and SYNTHIA to Cityscapes demonstrate the effectiveness of our method in domain adaptation of unsupervised semantic segmentation.

Details

ICML Conference 2020 Conference Paper

Learning to Learn Kernels with Variational Random Features

Xiantong Zhen
Haoliang Sun
Yingjun Du
Jun Xu 0019
Yilong Yin
Ling Shao 0001
Cees G. M. Snoek

We introduce kernels with random Fourier features in the meta-learning framework for few-shot learning. We propose meta variational random features (MetaVRF) to learn adaptive kernels for the base-learner, which is developed in a latent variable model by treating the random feature basis as the latent variable. We formulate the optimization of MetaVRF as a variational inference problem by deriving an evidence lower bound under the meta-learning framework. To incorporate shared knowledge from related tasks, we propose a context inference of the posterior, which is established by an LSTM architecture. The LSTM-based inference network can effectively integrate the context information of previous tasks with task-specific information, generating informative and adaptive features. The learned MetaVRF can produce kernels of high representational power with a relatively low spectral sampling rate and also enables fast adaptation to new tasks. Experimental results on a variety of few-shot regression and classification tasks demonstrate that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.

Details

ICML Conference 2020 Conference Paper

On the Number of Linear Regions of Convolutional Neural Networks

Huan Xiong
Lei Huang 0015
Mengyang Yu
Li Liu 0004
Fan Zhu 0001
Ling Shao 0001

One fundamental problem in deep learning is understanding the outstanding performance of deep Neural Networks (NNs) in practice. One explanation for the superiority of NNs is that they can realize a large class of complicated functions, i. e. , they have powerful expressivity. The expressivity of a ReLU NN can be quantified by the maximal number of linear regions it can separate its input space into. In this paper, we provide several mathematical results needed for studying the linear regions of CNNs, and use them to derive the maximal and average numbers of linear regions for one-layer ReLU CNNs. Furthermore, we obtain upper and lower bounds for the number of linear regions of multi-layer ReLU CNNs. Our results suggest that deeper CNNs have more powerful expressivity than their shallow counterparts, while CNNs have more expressivity than fully-connected NNs per parameter.

Details

ICRA Conference 2017 Conference Paper

DAP3D-Net: Where, what and how actions occur in videos?

Li Liu 0004
Yi Zhou 0007
Ling Shao 0001

Action parsing in videos with complex scenes is an interesting but challenging task in computer vision. In this paper, we propose a novel deep model based on 3D CNN (convolutional neural network) and LSTM (long short-term memory) module with a multi-task learning manner for effective Deep Action Parsing (DAP3D-Net) in videos. Particularly in the training phase, each action clip, sliced to several short consecutive segments, is fed into 3D CNN followed by LSTM to model the whole action dynamic information, so that action localization, classification and attributes learning can be jointly optimized via our deep model. Once the DAP3D-Net is trained, for an upcoming test video, we can describe each individual action in the video simultaneously as: Where the action occurs; What the action is and How the action is performed. To well demonstrate the effectiveness of the proposed DAP3D-Net, we also contribute a new Numerous-category Aligned Synthetic Action dataset, i. e. , NASA, which consists of 200, 000 action clips of 300 categories and with 33 pre-defined action attributes in two hierarchical levels (i. e. , low-level attributes of basic body part movements and high-level attributes related to action motion). We learn DAP3D-Net using the NASA dataset and then evaluate it on our collected Human Action Understanding (HAU) dataset and the public THUMOS dataset. Experimental results show that our approach can accurately localize, categorize and describe multiple actions in realistic videos.

Details

ICRA Conference 2017 Conference Paper

Semi-supervised vision-language mapping via variational learning

Yuming Shen
Li Zhang 0013
Ling Shao 0001

Understanding the semantic relations between vision and language data has become a research trend in artificial intelligence and robotic systems. The lack of training data is an essential issue for vision-language understanding. We address the problem of image and sentence cross-modal retrieval when paired training samples are not sufficient. Inspired by recent works in variational inference, in this paper, the autoencoding variational Bayes framework is novelly extended to a semi-supervised model for image-sentence mapping task. Our method does not require all training images and sentences to be paired. The proposed model is an end-to-end system, and consists of a two-level variational embedding structure where unpaired data are involved in the first level embedding to give support to intra-modality statistics so that the lower bound of the joint marginal likelihood of paired data embeddings can be better approximated. The proposed retrieval model is evaluated on two popular datasets, i. e. Flickr30K and Flickr8K, producing superior performances compared with related state-of-the-art methods.

Details

ICRA Conference 2016 Conference Paper

Arbitrary view action recognition via transfer dictionary learning on synthetic training data

Jingtian Zhang
Lining Zhang
Hubert P. H. Shum
Ling Shao 0001

Human action recognition is an important problem in robotic vision. Traditional recognition algorithms usually require the knowledge of view angle, which is not always available in robotic applications such as active vision. In this paper, we propose a new framework to recognize actions with arbitrary views. A main feature of our algorithm is that view-invariance is learned from synthetic 2D and 3D training data using transfer dictionary learning. This guarantees the availability of training data, and removes the hassle of obtaining real world video in specific viewing angles. The result of the process is a dictionary that can project real world 2D video into a view-invariant sparse representation. This facilitates the training of a view-invariant classifier. Experimental results on the IXMAS and N-UCLA datasets show significant improvements over existing algorithms.

Details

ICRA Conference 2016 Conference Paper

Color object recognition via cross-domain learning on RGB-D images

Yawen Huang
Fan Zhu 0001
Ling Shao 0001
Alejandro F. Frangi

This paper addresses the object recognition problem using multiple-domain inputs. We present a novel approach that utilizes labeled RGB-D data in the training stage, where depth features are extracted for enhancing the discriminative capability of the original learning system that only relies on RGB images. The highly dissimilar source and target domain data are mapped into a unified feature space through transfer at both feature and classifier levels. In order to alleviate cross-domain discrepancy, we employ a state-of-the-art domain-adaptive dictionary learning algorithm that updates image representations in both domains and the classifier parameters simultaneously. The proposed method is trained on a RGB-D Object dataset and evaluated on the Caltech-256 dataset. Experimental results suggest that our approach can lead to significant performance gain over the state-of-the-art methods.

Details