Arrow Research search

Author name cluster

Xiaogang Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

26 papers
2 author rows

Possible papers

26

ICRA Conference 2025 Conference Paper

Winding Number-Guided Edge-Preserving Implicit Neural Representation of CAD Surfaces

  • Yuhang Cheng
  • Zhiyuan Wang
  • Jialan He
  • Xiaogang Wang

Implicit surface representations have emerged as a powerful tool for the task of 3D reconstruction due to their excellent performance. Yet, when the normal information cannot be available, the previous methods often lead to unsatisfactory reconstruction results, even failure. To this end, we propose a winding number—guided implicit surface reconstruction method, which mainly consists of a winding number—guided regularizer and a dynamic edge sampling strategy. Among them, the winding number-guided regularizer can effectively constrain the global normal consistency of the input raw data, as well as improve the unsatisfactory implicit surface reconstruction result caused by the unavailability of normal information. Meanwhile, in order to reduce the excessive smoothing at sharp edges of implicit surface, we proposed a dynamic edge sampling strategy for sampling near the sharp edge regions of 3D shape, which can effectively avoid the regularizer from smoothing all regions. Finally, we combine them with a simple data term for robust implicit surface reconstruction. Compared with the state-of-the-art methods, experimental results show that our method significantly improves the quality of 3D reconstruction results. In addition, since the winding number-guided regularizer effectively constraints the globally consistent normal of the input 3D raw data, our method can also receive an additional gift, namely the globally consistent normal estimation results of 3D raw data.

Details

AAAI Conference 2024 Conference Paper

Cached Transformers: Improving Transformers with Differentiable Memory Cachde

  • Zhaoyang Zhang
  • Wenqi Shao
  • Yixiao Ge
  • Xiaogang Wang
  • Jinwei Gu
  • Ping Luo

This work introduces a new Transformer model called Cached Transformer, which uses Gated Recurrent Cached (GRC) attention to extend the self-attention mechanism with a differentiable memory cache of tokens. GRC attention enables attending to both past and current tokens, increasing the receptive field of attention and allowing for exploring long-range dependencies. By utilizing a recurrent gating unit to continuously update the cache, our model achieves significant advancements in \textbf{six} language and vision tasks, including language modeling, machine translation, ListOPs, image classification, object detection, and instance segmentation. Furthermore, our approach surpasses previous memory-based techniques in tasks such as language modeling and displays the ability to be applied to a broader range of situations.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Phased Consistency Models

  • Fu-Yun Wang
  • Zhaoyang Huang
  • Alexander W. Bergman
  • Dazhong Shen
  • Peng Gao
  • Michael Lingelbach
  • Keqiang Sun
  • Weikang Bian

Consistency Models (CMs) have made significant progress in accelerating the generation of diffusion models. However, their application to high-resolution, text-conditioned image generation in the latent space remains unsatisfactory. In this paper, we identify three key flaws in the current design of Latent Consistency Models~(LCMs). We investigate the reasons behind these limitations and propose Phased Consistency Models (PCMs), which generalize the design space and address the identified limitations. Our evaluations demonstrate that PCMs outperform LCMs across 1--16 step generation settings. While PCMs are specifically designed for multi-step refinement, they achieve comparable 1-step generation results to previously state-of-the-art specifically designed 1-step methods. Furthermore, we show the methodology of PCMs is versatile and applicable to video generation, enabling us to train the state-of-the-art few-step text-to-video generator. Our code is available at https: //github. com/G-U-N/Phased-Consistency-Model.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

A Unified Conditional Framework for Diffusion-based Image Restoration

  • Yi Zhang
  • Xiaoyu Shi
  • Dasong Li
  • Xiaogang Wang
  • Jian Wang
  • Hongsheng Li

Diffusion Probabilistic Models (DPMs) have recently shown remarkable performance in image generation tasks, which are capable of generating highly realistic images. When adopting DPMs for image restoration tasks, the crucial aspect lies in how to integrate the conditional information to guide the DPMs to generate accurate and natural output, which has been largely overlooked in existing works. In this paper, we present a unified conditional framework based on diffusion models for image restoration. We leverage a lightweight UNet to predict initial guidance and the diffusion model to learn the residual of the guidance. By carefully designing the basic module and integration module for the diffusion model block, we integrate the guidance and other auxiliary conditional information into every block of the diffusion model to achieve spatially-adaptive generation conditioning. To handle high-resolution images, we propose a simple yet effective inter-step patch-splitting strategy to produce arbitrary-resolution images without grid artifacts. We evaluate our conditional framework on three challenging tasks: extreme low-light denoising, deblurring, and JPEG restoration, demonstrating its significant improvements in perceptual quality and the generalization to restoration tasks. The code will be released at https: //zhangyi-3. github. io/project/UCDIR/.

PDF Details

NeurIPS Conference 2022 Conference Paper

Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs

  • Jinguo Zhu
  • Xizhou Zhu
  • Wenhai Wang
  • Xiaohua Wang
  • Hongsheng Li
  • Xiaogang Wang
  • Jifeng Dai

To build an artificial neural network like the biological intelligence system, recent works have unified numerous tasks into a generalist model, which can process various tasks with shared parameters and do not have any task-specific modules. While generalist models achieve promising results on various benchmarks, they have performance degradation on some tasks compared with task-specialized models. In this work, we find that interference among different tasks and modalities is the main factor to this phenomenon. To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models. Routing strategies under different levels of conditions are proposed to take both the training/inference cost and generalization ability into account. By incorporating the proposed Conditional MoEs, the recently proposed generalist model Uni-Perceiver can effectively mitigate the interference across tasks and modalities, and achieves state-of-the-art results on a series of downstream tasks via prompt tuning on 1% of downstream data. Moreover, the introduction of Conditional MoEs still holds the generalization ability of generalist models to conduct zero-shot inference on new tasks, e. g. , videotext retrieval and video caption. Code and pre-trained generalist models are publicly released at https: //github. com/fundamentalvision/Uni-Perceiver.

PDF Details

NeurIPS Conference 2021 Conference Paper

ReSSL: Relational Self-Supervised Learning with Weak Augmentation

  • Mingkai Zheng
  • Shan You
  • Fei Wang
  • Chen Qian
  • Changshui Zhang
  • Xiaogang Wang
  • Chang Xu

Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most of methods mainly focus on the instance level information (\ie, the different augmented images of the same instance should have the same feature or cluster into the same class), but there is a lack of attention on the relationships between different instances. In this paper, we introduced a novel SSL paradigm, which we term as relational self-supervised learning (ReSSL) framework that learns representations by modeling the relationship between different instances. Specifically, our proposed method employs sharpened distribution of pairwise similarities among different instances as \textit{relation} metric, which is thus utilized to match the feature embeddings of different augmentations. Moreover, to boost the performance, we argue that weak augmentations matter to represent a more reliable relation, and leverage momentum strategy for practical efficiency. Experimental results show that our proposed ReSSL significantly outperforms the previous state-of-the-art algorithms in terms of both performance and training efficiency.

PDF Details

EAAI Journal 2020 Journal Article

A complex process fault diagnosis method based on manifold distribution adaptation

  • Xiaogang Wang
  • Jianwei Zhao

The main challenge for complex industrial processes is that the distribution of new process data is different from that of old process data, resulting in poor performance when monitoring the new process (target domain) using the model established by the old process (source domain) data. Moreover, new process data is often unlabeled, which is more common in actual industrial processes. Therefore, this paper proposes a novel fault diagnosis method for complex industrial processes based on Manifold Distribution Adaptation (MDA) to cope with this challenge. Specifically, in order to avoid feature degradation caused by feature transformation directly in the original feature space, MDA first maps source domain data and target domain data to Grassmann manifold through geodesic flow kernel. Second, dynamic distribution adaptation and source domain instance re-weighting are performed simultaneously on the Grassmann manifold to reduce the shift between the domains. Then, the base classifier trained by the adaptive source domain data can achieve accurate classification of the target domain data. Finally, the superiority of MDA is verified on the public transfer learning datasets and the actual industrial process datasets.

Details DOI

AAAI Conference 2020 Conference Paper

KPNet: Towards Minimal Face Detector

  • Guanglu Song
  • Yu Liu
  • Yuhang Zang
  • Xiaogang Wang
  • Biao Leng
  • Qingsheng Yuan

The small receptive field and capacity of minimal neural networks limit their performance when using them to be the backbone of detectors. In this work, we find that the appearance feature of a generic face is discriminative enough for a tiny and shallow neural network to verify from the background. And the essential barriers behind us are 1) the vague definition of the face bounding box and 2) tricky design of anchor-boxes or receptive field. Unlike most topdown methods for joint face detection and alignment, the proposed KPNet detects small facial keypoints instead of the whole face by in the bottom-up manner. It first predicts the facial landmarks from a low-resolution image via the welldesigned fine-grained scale approximation and scale adaptive soft-argmax operator. Finally, the precise face bounding boxes, no matter how we define it, can be inferred from the keypoints. Without any complex head architecture or meticulous network designing, the KPNet achieves state-of-theart accuracy on generic face detection and alignment benchmarks with only ∼ 1M parameters, which runs at 1000fps on GPU and is easy to perform real-time on most modern frontend chips.

PDF Details

AAAI Conference 2020 Conference Paper

Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation

  • Yingjie Cai
  • Buyu Li
  • Zeyu Jiao
  • Hongsheng Li
  • Xingyu Zeng
  • Xiaogang Wang

Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images. Since the location recovery in 3D space is quite difficult on account of absence of depth information, this paper proposes a novel unified framework which decomposes the detection problem into a structured polygon prediction task and a depth recovery task. Different from the widely studied 2D bounding boxes, the proposed novel structured polygon in the 2D image consists of several projected surfaces of the target object. Compared to the widely-used 3D bounding box proposals, it is shown to be a better representation for 3D detection. In order to inversely project the predicted 2D structured polygon to a cuboid in the 3D physical world, the following depth recovery task uses the object height prior to complete the inverse projection transformation with the given camera projection matrix. Moreover, a fine-grained 3D box refinement scheme is proposed to further rectify the 3D detection results. Experiments are conducted on the challenging KITTI benchmark, in which our method achieves state-of-the-art detection accuracy.

PDF Details

NeurIPS Conference 2020 Conference Paper

PIE-NET: Parametric Inference of Point Cloud Edges

  • Xiaogang Wang
  • Yuelang Xu
  • Kai Xu
  • Andrea Tagliasacchi
  • Bin Zhou
  • Ali Mahdavi-Amiri
  • Hao Zhang

We introduce an end-to-end learnable technique to robustly identify feature edges in 3D point cloud data. We represent these edges as a collection of parametric curves (i. e. ,~lines, circles, and B-splines). Accordingly, our deep neural network, coined PIE-NET, is trained for parametric inference of edges. The network relies on a "region proposal" architecture, where a first module proposes an over-complete collection of edge and corner points, and a second module ranks each proposal to decide whether it should be considered. We train and evaluate our method on the ABC dataset, a large dataset of CAD models, and compare our results to those produced by traditional (non-learning) processing pipelines, as well as a recent deep learning based edge detector (EC-NET). Our results significantly improve over the state-of-the-art from both a quantitative and qualitative standpoint.

PDF Details

AAAI Conference 2019 Conference Paper

Gradient Harmonized Single-Stage Detector

  • Buyu Li
  • Yu Liu
  • Xiaogang Wang

Despite the great success of two-stage detectors, single-stage detector is still a more elegant and efficient way, yet suffers from the two well-known disharmonies during training, i. e. the huge difference in quantity between positive and negative examples as well as between easy and hard examples. In this work, we first point out that the essential effect of the two disharmonies can be summarized in term of the gradient. Further, we propose a novel gradient harmonizing mechanism (GHM) to be a hedging for the disharmonies. The philosophy behind GHM can be easily embedded into both classification loss function like cross-entropy (CE) and regression loss function like smooth-L1 (SL1) loss. To this end, two novel loss functions called GHM-C and GHM-R are designed to balancing the gradient flow for anchor classification and bounding box refinement, respectively. Ablation study on MS COCO demonstrates that without laborious hyper-parameter tuning, both GHM-C and GHM-R can bring substantial improvement for single-stage detector. Without any whistles and bells, the proposed model achieves 41. 6 mAP on COCO testdev set which surpass the state-of-the-art method, Focal Loss (FL) + SL1, by 0. 8. The code1 is released to facilitate future research.

PDF Details

NeurIPS Conference 2019 Conference Paper

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

  • Xihui Liu
  • Guojun Yin
  • Jing Shao
  • Xiaogang Wang
  • Hongsheng Li

Semantic image synthesis aims at generating photorealistic images from semantic layouts. Previous approaches with conditional generative adversarial networks (GAN) show state-of-the-art performance on this task, which either feed the semantic label maps as inputs to the generator, or use them to modulate the activations in normalization layers via affine transformations. We argue that convolutional kernels in the generator should be aware of the distinct semantic labels at different locations when generating images. In order to better exploit the semantic layout for the image generator, we propose to predict convolutional kernels conditioned on the semantic label map to generate the intermediate feature maps from the noise maps and eventually generate the images. Moreover, we propose a feature pyramid semantics-embedding discriminator, which is more effective in enhancing fine details and semantic alignments between the generated images and the input semantic layouts than previous multi-scale discriminators. We achieve state-of-the-art results on both quantitative metrics and subjective evaluation on various semantic segmentation datasets, demonstrating the effectiveness of our approach.

PDF Details

NeurIPS Conference 2019 Conference Paper

PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph

  • Yikang Li
  • Tao Ma
  • Yeqi BAI
  • Nan Duan
  • Sining Wei
  • Xiaogang Wang

Despite some exciting progress on high-quality image generation from structured (scene graphs) or free-form (sentences) descriptions, most of them only guarantee the image-level semantical consistency, i. e. the generated image matching the semantic meaning of the description. They still lack the investigations on synthesizing the images in a more controllable way, like finely manipulating the visual appearance of every object. Therefore, to generate the images with preferred objects and rich interactions, we propose a semi-parametric method, PasteGAN, for generating the image from the scene graph and the image crops, where spatial arrangements of the objects and their pair-wise relationships are defined by the scene graph and the object appearances are determined by the given object crops. To enhance the interactions of the objects in the output, we design a Crop Refining Network and an Object-Image Fuser to embed the objects as well as their relationships into one map. Multiple losses work collaboratively to guarantee the generated images highly respecting the crops and complying with the scene graphs while maintaining excellent image quality. A crop selector is also proposed to pick the most-compatible crops from our external object tank by encoding the interactions around the objects in the scene graph if the crops are not provided. Evaluated on Visual Genome and COCO-Stuff dataset, our proposed method significantly outperforms the SOTA methods on Inception Score, Diversity Score and Fréchet Inception Distance. Extensive experiments also demonstrate our method’s ability to generate complex and diverse images with given objects. The code is available at https: //github. com/yikang-li/PasteGAN.

PDF Details

AAAI Conference 2019 Conference Paper

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

  • Hang Zhou
  • Yu Liu
  • Ziwei Liu
  • Ping Luo
  • Xiaogang Wang

Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech. This is a challenging task because face appearance variation and semantics of speech are coupled together in the subtle movements of the talking face regions. Existing works either construct specific face appearance model on specific subjects or model the transformation between lip motion and speech. In this work, we integrate both aspects and enable arbitrary-subject talking face generation by learning disentangled audio-visual representation. We find that the talking face sequence is actually a composition of both subject-related information and speech-related information. These two spaces are then explicitly disentangled through a novel associative-and-adversarial training process. This disentangled representation has an advantage where both audio and video can serve as inputs for generation. Extensive experiments show that the proposed approach generates realistic talking face sequences on arbitrary subjects with much clearer lip motion patterns than previous work. We also demonstrate the learned audio-visual representation is extremely useful for the tasks of automatic lip reading and audio-video retrieval.

PDF Details

AAAI Conference 2019 Conference Paper

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

  • Mingyang Liang
  • Xiaoyang Guo
  • Hongsheng Li
  • Xiaogang Wang
  • You Song

Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any depth or disparity supervision. The estimated depth provides additional information complementary to original images, which can be helpful for other vision tasks such as tracking, recognition and detection. However, there are large appearance variations between images from different spectral bands, which is a challenge for cross-spectral stereo matching. Existing deep unsupervised stereo matching methods are sensitive to the appearance variations and do not perform well on cross-spectral data. We propose a novel unsupervised crossspectral stereo matching framework based on image-to-image translation. First, a style adaptation network transforms images across different spectral bands by cycle consistency and adversarial learning, during which appearance variations are minimized. Then, a stereo matching network is trained with image pairs from the same spectra using view reconstruction loss. At last, the estimated disparity is utilized to supervise the spectral translation network in an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is proposed to improve the robustness of spectral translation. Our method can tackle appearance variations and enhance the robustness of unsupervised cross-spectral stereo matching. Experimental results show that our method achieves good performance without using depth supervision or explicit semantic information.

PDF Details

AAAI Conference 2018 Conference Paper

Co-Attending Free-Form Regions and Detections With Multi-Modal Multiplicative Feature Embedding for Visual Question Answering

  • Pan Lu
  • Hongsheng Li
  • Wei Zhang
  • Jianyong Wang
  • Xiaogang Wang

Recently, the Visual Question Answering (VQA) task has gained increasing attention in artificial intelligence. Existing VQA methods mainly adopt the visual attention mechanism to associate the input question with corresponding image regions for effective question answering. The free-form region based and the detection-based visual attention mechanisms are mostly investigated, with the former ones attending free-form image regions and the latter ones attending prespecified detection-box regions. We argue that the two attention mechanisms are able to provide complementary information and should be effectively integrated to better solve the VQA problem. In this paper, we propose a novel deep neural network for VQA that integrates both attention mechanisms. Our proposed framework effectively fuses features from free-form image regions, detection boxes, and question representations via a multi-modal multiplicative feature embedding scheme to jointly attend question-related free-form image regions and detection boxes for more accurate question answering. The proposed method is extensively evaluated on two publicly available datasets, COCO-QA and VQA, and outperforms state-of-the-art approaches. Source code is available at https: //github. com/lupantech/dual-mfa-vqa.

PDF Details

NeurIPS Conference 2018 Conference Paper

FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification

  • Yixiao Ge
  • Zhuowan Li
  • Haiyu Zhao
  • Guojun Yin
  • Shuai Yi
  • Xiaogang Wang
  • Hongsheng Li

Person re-identification (reID) is an important task that requires to retrieve a person's images from an image dataset, given one image of the person of interest. For learning robust person features, the pose variation of person images is one of the key challenges. Existing works targeting the problem either perform human alignment, or learn human-region-based representations. Extra pose information and computational cost is generally required for inference. To solve this issue, a Feature Distilling Generative Adversarial Network (FD-GAN) is proposed for learning identity-related and pose-unrelated representations. It is a novel framework based on a Siamese structure with multiple novel discriminators on human poses and identities. In addition to the discriminators, a novel same-pose loss is also integrated, which requires appearance of a same person's generated images to be similar. After learning pose-unrelated person features with pose guidance, no auxiliary pose information and additional computational cost is required during testing. Our proposed FD-GAN achieves state-of-the-art performance on three person reID datasets, which demonstrates that the effectiveness and robust feature distilling capability of the proposed FD-GAN.

PDF Details

AAAI Conference 2018 Conference Paper

Spatial as Deep: Spatial CNN for Traffic Scene Understanding

  • Xingang Pan
  • Jianping Shi
  • Ping Luo
  • Xiaogang Wang
  • Xiaoou Tang

Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-byslice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset1. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8. 7% and 4. 6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96. 53%.

PDF Details

NeurIPS Conference 2017 Conference Paper

Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction

  • Dan Xu
  • Wanli Ouyang
  • Xavier Alameda-Pineda
  • Elisa Ricci
  • Xiaogang Wang
  • Nicu Sebe

Recent works have shown that exploiting multi-scale representations deeply learned via convolutional neural networks (CNN) is of tremendous importance for accurate contour detection. This paper presents a novel approach for predicting contours which advances the state of the art in two fundamental aspects, i. e. multi-scale feature generation and fusion. Different from previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, we introduce a hierarchical deep model which produces more rich and complementary representations. Furthermore, to refine and robustly fuse the representations learned at different scales, the novel Attention-Gated Conditional Random Fields (AG-CRFs) are proposed. The experiments ran on two publicly available datasets (BSDS500 and NYUDv2) demonstrate the effectiveness of the latent AG-CRF model and of the overall hierarchical framework.

PDF Details

NeurIPS Conference 2016 Conference Paper

CRF-CNN: Modeling Structured Information in Human Pose Estimation

  • Xiao Chu
  • Wanli Ouyang
  • Hongsheng Li
  • Xiaogang Wang

Deep convolutional neural networks (CNN) have achieved great success. On the other hand, modeling structural information has been proved critical in many vision problems. It is of great interest to integrate them effectively. In a classical neural network, there is no message passing between neurons in the same layer. In this paper, we propose a CRF-CNN framework which can simultaneously model structural information in both output and hidden feature layers in a probabilistic way, and it is applied to human pose estimation. A message passing scheme is proposed, so that in various layers each body joint receives messages from all the others in an efficient way. Such message passing can be implemented with convolution between features maps in the same layer, and it is also integrated with feedforward propagation in neural networks. Finally, a neural network implementation of end-to-end learning CRF-CNN is provided. Its effectiveness is demonstrated through experiments on two benchmark datasets.

PDF Details

AAAI Conference 2016 Conference Paper

Face Model Compression by Distilling Knowledge from Neurons

  • Ping Luo
  • Zhenyao Zhu
  • Ziwei Liu
  • Xiaogang Wang
  • Xiaoou Tang

The recent advanced face recognition systems were built on large Deep Neural Networks (DNNs) or their ensembles, which have millions of parameters. However, the expensive computation of DNNs make their deployment difficult on mobile and embedded devices. This work addresses model compression for face recognition, where the learned knowledge of a large teacher network or its ensemble is utilized as supervision to train a compact student network. Unlike previous works that represent the knowledge by the soften label probabilities, which are difficult to fit, we represent the knowledge by using the neurons at the higher hidden layer, which preserve as much information as the label probabilities, but are more compact. By leveraging the essential characteristics (domain knowledge) of the learned face representation, a neuron selection method is proposed to choose neurons that are most relevant to face recognition. Using the selected neurons as supervision to mimic the single networks of DeepID2+ and DeepID3, which are the state-of-the-art face recognition systems, a compact student with simple network structure achieves better verification accuracy on LFW than its teachers, respectively. When using an ensemble of DeepID2+ as teacher, a mimicked student is able to outperform it and achieves 51. 6× compression ratio and 90× speed-up in inference, making this cumbersome model applicable on portable devices.

PDF Details

NeurIPS Conference 2014 Conference Paper

Deep Learning Face Representation by Joint Identification-Verification

  • Yi Sun
  • Yuheng Chen
  • Xiaogang Wang
  • Xiaoou Tang

The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 features extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 features extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset, 99. 15% face verification accuracy is achieved. Compared with the best previous deep learning result on LFW, the error rate has been significantly reduced by 67%.

PDF Details

NeurIPS Conference 2014 Conference Paper

Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations

  • Zhenyao Zhu
  • Ping Luo
  • Xiaogang Wang
  • Xiaoou Tang

Various factors, such as identities, views (poses), and illuminations, are coupled in face images. Disentangling the identity and view representations is a major challenge in face recognition. Existing face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. This is different from the behavior of human brain. Intriguingly, even without accessing 3D data, human not only can recognize face identity, but can also imagine face images of a person under different viewpoints given a single 2D image, making face perception in the brain robust to view changes. In this sense, human brain has learned and encoded 3D face models from 2D images. To take into account this instinct, this paper proposes a novel deep neural net, named multi-view perceptron (MVP), which can untangle the identity and view features, and infer a full spectrum of multi-view images in the meanwhile, given a single 2D face image. The identity features of MVP achieve superior performance on the MultiPIE dataset. MVP is also capable to interpolate and predict images under viewpoints that are unobserved in the training data.

PDF Details

IJCAI Conference 2013 Conference Paper

Dimensionality Reduction with Generalized Linear Models

  • Mo Chen
  • Wei Li
  • Wei Zhang
  • Xiaogang Wang

In this paper, we propose a general dimensionality reduction method for data generated from a very broad family of distributions and nonlinear functions based on the generalized linear model, called Generalized Linear Principal Component Analysis (GLPCA). Data of different domains often have very different structures. These data can be modeled by different distributions and reconstruction functions. For example, real valued data can be modeled by the Gaussian distribution with a linear reconstruction function, whereas binary valued data may be more appropriately modeled by the Bernoulli distribution with a logit or probit function. Based on general linear models, we propose a unified framework for extracting features from data of different domains. A general optimization algorithm based on natural gradient ascent on distribution manifold is proposed for obtaining the maximum likelihood solutions. We also present some specific algorithms derived from this framework to deal with specific data modeling problems such as document modeling. Experimental results of these algorithms on several data sets are shown for the validation of GLPCA.

PDF Details DOI

YNIMG Journal 2011 Journal Article

Tractography segmentation using a hierarchical Dirichlet processes mixture model

  • Xiaogang Wang
  • W. Eric L. Grimson
  • Carl-Fredrik Westin

In this paper, we propose a new nonparametric Bayesian framework to cluster white matter fiber tracts into bundles using a hierarchical Dirichlet processes mixture (HDPM) model. The number of clusters is automatically learned driven by data with a Dirichlet process (DP) prior instead of being manually specified. After the models of bundles have been learned from training data without supervision, they can be used as priors to cluster/classify fibers of new subjects for comparison across subjects. When clustering fibers of new subjects, new clusters can be created for structures not observed in the training data. Our approach does not require computing pairwise distances between fibers and can cluster a huge set of fibers across multiple subjects. We present results on several data sets, the largest of which has more than 120, 000 fibers.

Details DOI

NeurIPS Conference 2007 Conference Paper

Spatial Latent Dirichlet Allocation

  • Xiaogang Wang
  • Eric Grimson

In recent years, the language model Latent Dirichlet Allocation (LDA), which clusters co-occurring words into topics, has been widely appled in the computer vision field. However, many of these applications have difficulty with modeling the spatial and temporal structure among visual words, since LDA assumes that a document is a bag-of-words''. It is also critical to properly design words'' and “documents” when using a language model to solve vision problems. In this paper, we propose a topic model Spatial Latent Dirichlet Allocation (SLDA), which better encodes spatial structure among visual words that are essential for solving many vision problems. The spatial information is not encoded in the value of visual words but in the design of documents. Instead of knowing the partition of words into documents \textit{a priori}, the word-document assignment becomes a random hidden variable in SLDA. There is a generative procedure, where knowledge of spatial structure can be flexibly added as a prior, grouping visual words which are close in space into the same document. We use SLDA to discover objects from a collection of images, and show it achieves better performance than LDA.

PDF Details
v2026.09.13