Author name cluster

Mao Ye

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

27 papers

2 author rows

AAAI Conference 2026 Conference Paper

Cross-domain Joint Learning with Prototype-guided Mixture-of-Experts for Infrared Moving Small Target Detection

Weiwei Duan
Luping Ji
Jianghong Huang
Sicheng Zhu
Mao Ye

Infrared small target detection often faces significant domain gaps across datasets due to varying sensors and scene distributions. Currently, most existing methods are typically based on single-domain learning (i.e., training and test are on the same dataset), requiring training separate detectors when considering different datasets. However, they overlook the valuable public knowledge across domains and limit the applicability in multiple infrared scenarios. To break through single-domain learning, implementing only one universal detector simultaneously on multiple datasets, as the first exploration, we propose a cross-domain joint learning task framework with prototype-guided Mixture-of-Experts (CoMoE). Specifically, it designs a hyperspherical prototype learning to adaptively maintain both domain-specific prototypes and global prototypes, enhancing cross-domain feature representation. Meanwhile, a domain-aware Mixture-of-Experts with Top-K routing strategy is proposed to select the optimal domain experts. Moreover, to enhance cross-domain feature alignment, we design an adaptive cross-domain feature modulation with noise-guided contrastive learning. The extensive experiments on a newly constructed benchmark comprising three datasets verify the superiority of our CoMoE, even under limited data settings. It could often surpass general joint learning methods, and state-of-the-art (SOTA) single-domain ones.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Domain-Auxiliary Infrared Moving Small Target Detection by Learning to Overlook Domain Discrepancy

Shengjia Chen
Luping Ji
Shuang Peng
Sicheng Zhu
Mao Ye

Currently, almost all traditional infrared small target detection methods work on the assumption that training and test sets always belong to the same domain, and training samples are sufficient. However, in real applications, a new detection task could often have no sufficient training samples from a special domain. In this situation, adopting the auxiliary data from big-sample domains is usually believed to be one of the most potential solutions. However, exceeding expectations, it is found that simply adding auxiliary samples cannot often be always effective, even causing performance decline, due to existing infrared domain shift. To overcome this unexpected problem, we propose the first infrared moving small target detection framework with domain-auxiliary supports by Learning to Overlook Domain Discrepancy (Loddis). This framework consists of three primary processing stages: correlation weakening, domain confusing, and target consistency contrastive learning. Breaking through traditional learning paradigm, through auxiliary data, it enables the model to focus more on targets themselves, and less on image backgrounds, minimizing the sensitivity to domain discrepancy. The extensive experiments on 6 different-domain datasets show the effectiveness and superiority of the proposed Loddis framework for infrared small target detection.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Hierarchical Frequency-Guided Alignment Transformer for Compressed Video Quality Enhancement

Liuhan Peng
Shuai Li
Yanbo Gao
Mao Ye
Chong Lv

During the video encoding process, the original spatial domain signal is first transformed into the frequency domain, followed by quantization and compression. As a result, the quality degradation in compressed videos primarily stems from distortions in the frequency domain information. However, existing video enhancement methods typically directly fuse information from adjacent frames in the spatial domain, making it difficult for models to effectively compensate for frequency domain distortions, which leads to suboptimal detail restoration. To address this issue, we propose a Hierarchical Frequency-Guided Alignment Transformer. Additionally, by analyzing the characteristics of the frequency domain, we find that different frequency bands exhibit both correlations and a certain degree of independence. Based on this, we introduce a Frequency-Aware Transformer module that employs a combination of independent and mixed processing to optimize information exchange across different frequency domains, effectively mitigating cross-interference from irrelevant information. Experimental results demonstrate that, compared to existing methods, our approach achieves state-of-the-art performance in objective metrics (PSNR/SSIM), perceptual quality (LPIPS), and subjective visual effects, while reducing model complexity.

PDF Details DOI

EAAI Journal 2026 Journal Article

Multi-exposure high dynamic range reconstruction by incorporating imaging knowledge

Hu Wang
Mao Ye
Dengyan Luo
Yan Gan

The existing photographic equipment is not able to capture scenes of the natural world very well. Thus, the problem of reconstructing high dynamic range (HDR) images from multi-exposure low dynamic range (LDR) images arises because these images have different details. The existing methods do not fully leverage imaging knowledge in the LDR image generation pipeline, resulting in design redundancy and inefficient resource utilization. We propose a new Multi-Exposure HDR reconstruction by incorporating Imaging Knowledge (MEIK) for efficient HDR image reconstruction. Our method consists of two parts: fusion of LDR features and reconstruction of HDR feature. Due to object motion and exposure time effects, LDR features with different exposures need to be fused. A Multi-Exposure Information Aggregation (MEIA) module is proposed to fuse LDR features based on Mamba. After that, an Inverse imaging Knowledge-Driven (IKD) cluster is employed to reconstruct the HDR feature, which is a cascade of IKD blocks at different scales. The IKD block consists of three parts: HDR information recovery, imaging parameter adjustment, and noise suppression, used to simulate the mathematical formula for multi-exposure HDR imaging. Experimental results demonstrate that the proposed MEIK model outperforms existing state-of-the-art models and exhibits strong scalability.

Details DOI

AAAI Conference 2025 Conference Paper

FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments

Aoru Xue
Yiming Ren
Zining Song
Mao Ye
Xinge Zhu
Yuexin Ma

We propose a novel hybrid calibration-free method FreeCap to accurately capture global multi-person motions in open environments. Our system combines a single LiDAR with expandable moving cameras, allowing for flexible and precise motion estimation in a unified world coordinate. In particular, We introduce a local-to-global pose-aware cross-sensor human-matching module that predicts the alignment among each sensor, even in the absence of calibration. Additionally, our coarse-to-fine sensor-expandable pose optimizer further optimizes the 3D human key points and the alignments, it is also capable of incorporating additional cameras to enhance accuracy. Extensive experiments on Human-M3 and FreeMotion datasets demonstrate that our method significantly outperforms state-of-the-art single-modal methods, offering an expandable and efficient solution for multi-person motion capture across various applications.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Motion Prior Knowledge Learning with Homogeneous Language Descriptions for Moving Infrared Small Target Detection

Shengjia Chen
Luping Ji
Weiwei Duan
Shuang Peng
Mao Ye

Different from traditional object detection, pure vision is not enough to infrared small target detection, due to small target size and weak background contrast. For promoting detection performance, more target representations are needed. Currently, motion representations have been proved to be one of the most potential feature kinds for infrared small target detection. Existing methods have an obvious weakness, that besides vision features, they could only capture coarse motion representations from temporal domain. With vision features, fine motion representations could be more effective to enhance detection performance. To overcome this weakness, inspired by prevalent vision-language models, we propose the first vision-language framework with motion prior knowledge learning (MoPKL). Breaking through traditional pure-vision modality, it utilizes homogeneous language descriptions, formatted for moving targets, to directionally guide vision channel learning motion prior knowledge. With the facilitation of motion-vision alignment and motion-relation mining, the motion of infrared small targets is further refined by graph attention, to generate more fine motion representations. The extensive experiments on datasets ITSDT-15K and IRDST show that our framework is effective. It could often obviously outperform other methods.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Multimodal Causal Reasoning for UAV Object Detection

Nianxin Li
Mao Ye
Lihua Zhou
Shuaifeng Li
Song Tang
Luping Ji
Ce Zhu

Unmanned Aerial Vehicle (UAV) object detection faces significant challenges due to complex environmental conditions and different imaging conditions. These factors introduce significant changes in scale and appearance, particularly for small objects that occupy limited pixels and exhibit limited information, complicating detection tasks. To address these challenges, we propose a Multimodel Causal Reasoning framework based on YOLO backbone for UAV Object Detection (MCR-UOD). The key idea is to use the backdoor adjustment to discover the condition-invariant object representation for easy detection. Specifically, the YOLO backbone is first adjusted to incorporate the pre-trained vision-language model. The original category labels are replaced with semantic text prompts, and the detection head is replaced with text-image contrastive learning. Based on this backbone, our method consists of two parts. The first part, named language guided region exploration, discovers the regions with high probability of object existence using text embeddings based on vision-language model such as CLIP. Another part is the backdoor adjustment casual reasoning module, which constructs a confounder dictionary tailored to different imaging conditions to capture global image semantics and derives a prior probability distribution of shooting conditions. During causal inference, we use the confounder dictionary and the prior to intervene on local instance features, disentangling condition variations, and obtaining condition-invariant representations. Experimental results on several public datasets confirm the state-of-the-art performance of our approach. The code, data and models will be released upon publication of this paper.

PDF Details

AAAI Conference 2025 Conference Paper

Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classification

Jiaxiang Gou
Luping Ji
Pei Liu
Mao Ye

Whole Slide Image (WSI) classification has very significant applications in clinical pathology, e.g., tumor identification and cancer diagnosis. Currently, most research attention is focused on Multiple Instance Learning (MIL) using static datasets. One of the most obvious weaknesses of these methods is that they cannot efficiently preserve and utilize previously learned knowledge. With any new data arriving, classification models are required to be re-trained on both previous and current new data. To overcome this shortcoming and break through traditional vision modality, this paper proposes the first Vision-Language-based framework with Queryable Prototype Multiple Instance Learning (QPMIL-VL) specially designed for incremental WSI classification. This framework mainly consists of two information processing branches: one is for generating bag-level features by prototype-guided aggregation of instance features, while the other is for enhancing class features through a combination of class ensemble, tunable vector and class similarity loss. The experiments on four public WSI datasets demonstrate that our QPMIL-VL framework is effective for incremental WSI classification and often significantly outperforms other compared methods, achieving state-of-the-art (SOTA) performance.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Self-Prompting Analogical Reasoning for UAV Object Detection

Nianxin Li
Mao Ye
Lihua Zhou
Song Tang
Yan Gan
Zizhuo Liang
Xiatian Zhu

Unmanned Aerial Vehicle Object Detection (UAVOD) presents unique challenges due to varying altitudes, dynamic backgrounds, and the small size of objects. Traditional detection methods often struggle with these challenges, as they typically rely on visual feature only and fail to extract the semantic relations between the objects. To address these limitations, we propose a novel approach named Self-Prompting Analogical Reasoning (SPAR). Our method utilizes the vision-language model (CLIP) to generate context-aware prompts based on image feature, providing rich semantic information that guides analogical reasoning. SPAR includes two main modules: self-prompting and analogical reasoning. Self-prompting module based on learnable description and CLIP-text encoder generates context-aware prompt by combining specific image feature; then an objectness prompt score map is produced by computing the similarity between pixel-level features and context-aware prompt. With this score map, multi-scale image features are enhanced and pixel-level features are chosen for graph construction. While for analogical reasoning module, graph nodes consists of category-level prompt nodes and pixel-level image feature nodes. Analogical inference is based graph convolution. Under the guidance of category-level nodes, different-scale object features have been enhanced, which helps achieve more accurate detection of challenging objects. Extensive experiments illustrate that SPAR outperforms traditional methods, offering a more robust and accurate solution for UAVOD.

PDF Details DOI

ICRA Conference 2025 Conference Paper

Uncertainty-Guided Enhancement on Driving Perception System Via Foundation Models

Yunhao Yang
Yuxin Hu
Mao Ye
Zaiwei Zhang
Zhichao Lu
Yi Xu
Ufuk Topcu
Ben Snyder

Multimodal foundation models offer promising advancements for enhancing driving perception systems, but their high computational and financial costs pose challenges. We develop a method that leverages foundation models to refine predictions from existing driving perception modelssuch as enhancing object classification accuracy-while minimizing the frequency of using these resource-intensive models. The method quantitatively characterizes uncertainties in the perception model's predictions and engages the foundation model only when these uncertainties exceed a pre-specified threshold. Specifically, it characterizes uncertainty by calibrating the perception model's confidence scores into theoretical lower bounds on the probability of correct predictions using conformal prediction. Then, it sends images to the foundation model and queries for refining the predictions only if the theoretical bound of the perception model's outcome is below the threshold. Additionally, we propose a temporal inference mechanism that enhances prediction accuracy by integrating historical predictions, leading to tighter theoretical bounds. The method demonstrates a 10 to 15 percent improvement in prediction accuracy and reduces the number of queries to the foundation model by 50 percent, based on quantitative evaluations from driving datasets.

Details

NeurIPS Conference 2024 Conference Paper

Cloud Object Detector Adaptation by Integrating Different Source Knowledge

Shuaifeng Li
Mao Ye
Lihua Zhou
Nianxin Li
Siying Xiao
Song Tang
Xiatian Zhu

We propose to explore an interesting and promising problem, Cloud Object Detector Adaptation (CODA), where the target domain leverages detections provided by a large cloud model to build a target detector. Despite with powerful generalization capability, the cloud model still cannot achieve error-free detection in a specific target domain. In this work, we present a novel Cloud Object detector adaptation method by Integrating different source kNowledge (COIN). The key idea is to incorporate a public vision-language model (CLIP) to distill positive knowledge while refining negative knowledge for adaptation by self-promotion gradient direction alignment. To that end, knowledge dissemination, separation, and distillation are carried out successively. Knowledge dissemination combines knowledge from cloud detector and CLIP model to initialize a target detector and a CLIP detector in target domain. By matching CLIP detector with the cloud detector, knowledge separation categorizes detections into three parts: consistent, inconsistent and private detections such that divide-and-conquer strategy can be used for knowledge distillation. Consistent and private detections are directly used to train target detector; while inconsistent detections are fused based on a consistent knowledge generation network, which is trained by aligning the gradient direction of inconsistent detections to that of consistent detections, because it provides a direction toward an optimal target detector. Experiment results demonstrate that the proposed COIN method achieves the state-of-the-art performance.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

Independent Feature Decomposition and Instance Alignment for Unsupervised Domain Adaptation

Qichen He
Siying Xiao
Mao Ye
Xiatian Zhu
Ferrante Neri
Dongde Hou

Existing Unsupervised Domain Adaptation (UDA) methods typically attempt to perform knowledge transfer in a domain-invariant space explicitly or implicitly. In practice, however, the obtained features is often mixed with domain-specific information which causes performance degradation. To overcome this fundamental limitation, this article presents a novel independent feature decomposition and instance alignment method (IndUDA in short). Specifically, based on an invertible flow, we project the base features into a decomposed latent space with domain-invariant and domain-specific dimensions. To drive semantic decomposition independently, we then swap the domain-invariant part across source and target domain samples with the same category and require their inverted features are consistent in class-level with the original features. By treating domain-specific information as noise, we replace it by Gaussian noise and further regularize source model training by instance alignment, i. e. , requiring the base features close to the corresponding reconstructed features, respectively. Extensive experiment results demonstrate that our method achieves state-of-the-art performance on popular UDA benchmarks. The appendix and code are available at https: //github. com/ayombeach/IndUDA.

PDF Details DOI

EAAI Journal 2023 Journal Article

Single-image HDR reconstruction by dual learning the camera imaging process

Lei She
Mao Ye
Shuai Li
Yu Zhao
Ce Zhu
Hu Wang

It is a very challenging problem to reconstruct a high dynamic range (HDR) image from a single exposure image. There exist three problems, i. e. , the many-to-many mapping problem between low dynamic range (LDR) images and HDR images, the image quality problem caused by the change of dynamic range and the problem of unpaired LDR–HDR training images. These problems can be solved to some extent through a dual learning framework simultaneously to learn the forward and reverse of camera imaging processes. This procedure is divided into a primary module, to reconstruct HDR from LDR, and a secondary module to reversely mapping the HDR to LDR. The secondary module guides the learning of primary module by constraining the outputs of the primary module. After that, the attention mechanism is used to solve the problem of unnatural perception caused by the change of dynamic range. In the end, with the advantage of our dual learning framework, unpaired data is further explored to train our model, which enriches the training samples. Compared with the state-of-the-art methods, a large number of quantitative and qualitative experiments confirm that our method can achieve better performance.

Details DOI

NeurIPS Conference 2022 Conference Paper

BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach

Bo Liu
Mao Ye
Stephen Wright
Peter Stone
Qiang Liu

Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning. Conventional BO methods need to differentiate through the low-level optimization process with implicit differentiation, which requires expensive calculations related to the Hessian matrix. There has been a recent quest for first-order methods for BO, but the methods proposed to date tend to be complicated and impractical for large-scale deep learning applications. In this work, we propose a simple first-order BO algorithm that depends only on first-order gradient information, requires no implicit differentiation, and is practical and efficient for large-scale non-convex functions in deep learning. We provide non-asymptotic convergence analysis of the proposed method to stationary points for non-convex objectives and present empirical results that show its superior practical performance.

PDF Details

NeurIPS Conference 2022 Conference Paper

Diffusion-based Molecule Generation with Informative Prior Bridges

Lemeng Wu
Chengyue Gong
Xingchao Liu
Mao Ye
Qiang Liu

AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development. Because the molecules are governed by physical laws, a key challenge is to incorporate prior information into the training procedure to generate high-quality and realistic molecules. We propose a simple and novel approach to steer the training of diffusion-based generative models with physical and statistics prior information. This is achieved by constructing physically informed diffusion bridges, stochastic processes that guarantee to yield a given observation at the fixed terminal time. We develop a Lyapunov function based method to construct and determine bridges, and propose a number of proposals of informative prior bridges for both high-quality molecule generation and uniformity-promoted 3D point cloud generation. With comprehensive experiments, we show that our method provides a powerful approach to the 3D generation task, yielding molecule structures with better quality and stability scores and more uniformly distributed point clouds of high qualities.

PDF Details

NeurIPS Conference 2022 Conference Paper

First Hitting Diffusion Models for Generating Manifold, Graph and Categorical Data

Mao Ye
Lemeng Wu
Qiang Liu

We propose a family of First Hitting Diffusion Models (FHDM), deep generative models that generate data with a diffusion process that terminates at a random first hitting time. This yields an extension of the standard fixed-time diffusion models that terminate at a pre-specified deterministic time. Although standard diffusion models are designed for continuous unconstrained data, FHDM is naturally designed to learn distributions on continuous as well as a range of discrete and structure domains. Moreover, FHDM enables instance-dependent terminate time and accelerates the diffusion process to sample higher quality data with fewer diffusion steps. Technically, we train FHDM by maximum likelihood estimation on diffusion trajectories augmented from observed data with conditional first hitting processes (i. e. , bridge) derived based on Doob's $h$-transform, deviating from the commonly used time-reversal mechanism. We apply FHDM to generate data in various domains such as point cloud (general continuous distribution), climate and geographical events on earth (continuous distribution on the sphere), unweighted graphs (distribution of binary matrices), and segmentation maps of 2D images (high-dimensional categorical distribution). We observe considerable improvement compared with the state-of-the-art approaches in both quality and speed.

PDF Details

IJCAI Conference 2022 Conference Paper

KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction

Hu Wang
Mao Ye
Xiatian Zhu
Shuai Li
Ce Zhu
Xue Li

Recently, with the rise of high dynamic range (HDR) display devices, there is a great demand to transfer traditional low dynamic range (LDR) images into HDR versions. The key to success is how to solve the many-to-many mapping problem. However, the existing approaches either do not consider constraining solution space or just simply imitate the inverse camera imaging pipeline in stages, without directly formulating the HDR image generation process. In this work, we address this problem by integrating LDR-to-HDR imaging knowledge into an UNet architecture, dubbed as Knowledge-inspired UNet (KUNet). The conversion from LDR-to-HDR image is mathematically formulated, and can be conceptually divided into recovering missing details, adjusting imaging parameters and reducing imaging noise. Accordingly, we develop a basic knowledge-inspired block (KIB) including three subnetworks corresponding to the three procedures in this HDR imaging process. The KIB blocks are cascaded in the similar way to the UNet to construct HDR image with rich global information. In addition, we also propose a knowledge inspired jump-connect structure to fit a dynamic range gap between HDR and LDR images. Experimental results demonstrate that the proposed KUNet achieves superior performance compared with the state-of-the-art methods. The code, dataset and appendix materials are available at https: //github. com/wanghu178/KUNet. git.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

MetaTeacher: Coordinating Multi-Model Domain Adaptation for Medical Image Classification

Zhenbin Wang
Mao Ye
Xiatian Zhu
Liuhan Peng
Liang Tian
Yingying Zhu

In medical image analysis, we often need to build an image recognition system for a target scenario with the access to small labeled data and abundant unlabeled data, as well as multiple related models pretrained on different source scenarios. This presents the combined challenges of multi-source-free domain adaptation and semi-supervised learning simultaneously. However, both problems are typically studied independently in the literature, and how to effectively combine existing methods is non-trivial in design. In this work, we introduce a novel MetaTeacher framework with three key components: (1) A learnable coordinating scheme for adaptive domain adaptation of individual source models, (2) A mutual feedback mechanism between the target model and source models for more coherent learning, and (3) A semi-supervised bilevel optimization algorithm for consistently organizing the adaption of source models and the learning of target model. It aims to leverage the knowledge of source models adaptively whilst maximize their complementary benefits collectively to counter the challenge of limited supervision. Extensive experiments on five chest x-ray image datasets show that our method outperforms clearly all the state-of-the-art alternatives. The code is available at https: //github. com/wongzbb/metateacher.

PDF Details

ICRA Conference 2022 Conference Paper

Multi-Class 3D Object Detection with Single-Class Supervision

Mao Ye
Chenxi Liu 0001
Maoqing Yao
Weiyue Wang 0002
Zhaoqi Leng
Charles R. Qi
Dragomir Anguelov

While multi-class 3D detectors are needed in many robotics applications, training them with fully labeled datasets can be expensive in labeling cost. An alternative approach is to have targeted single-class labels on disjoint data samples. In this paper, we are interested in training a multi-class 3D object detection model, while using these single-class labeled data. We begin by detailing the unique stance of our “Single-Class Supervision” (SCS) setting with respect to related concepts such as partial supervision and semi supervision. Then, based on the case study of training the multi-class version of Range Sparse Net (RSN), we adapt a spectrum of algorithms - from supervised learning to pseudo-labeling - to fully exploit the properties of our SCS setting, and perform extensive ablation studies to identify the most effective algorithm and practice. Empirical experiments on the Waymo Open Dataset show that proper training under SCS can approach or match full supervision training while saving labeling costs.

Details

NeurIPS Conference 2021 Conference Paper

argmax centroid

Chengyue Gong
Mao Ye
Qiang Liu

We propose a general method to construct centroid approximation for the distribution of maximum points of a random function (a. k. a. argmax distribution), which finds broad applications in machine learning. Our method optimizes a set of centroid points to compactly approximate the argmax distribution with a simple objective function, without explicitly drawing exact samples from the argmax distribution. Theoretically, the argmax centroid method can be shown to minimize a surrogate of Wasserstein distance between the ground-truth argmax distribution and the centroid approximation under proper conditions. We demonstrate the applicability and effectiveness of our method on a variety of real-world multi-task learning applications, including few-shot image classification, personalized dialogue systems and multi-target domain adaptation.

PDF Details

EAAI Journal 2021 Journal Article

Local–Global Attentive Adaptation for Object Detection

Dan Zhang
Jingjing Li
Xingpeng Li
Zhekai Du
Lin Xiong
Mao Ye

Adversarial adaptive methods have been proven to be useful for domain transfer in many fields such as image recognition and semantic segmentation, etc However, for object detection, since each image could have different combinations of objects, brutally aligning all the images without considering their transferability may cause the notorious phenomena named ‘negative transfer’. On the other hand, strong matching the local-level features makes sense, as it not only reduces the discrepancy between different domain distributions, but preserves the category-level semantic information. However, it is hard to markedly achieve domain invariance using a simple adversarial adaptive method. In this work, we propose an effective method termed Local–Global Attentive Adaptation for object Detection (LGAAD). Our method can alleviate the negative transfer caused by improper global alignments through leveraging an adaptively and dynamically weighted transferability to highlight the more transferable images. Furthermore, the proposed method also achieves the strong matching between two domains at local-level features to alleviate the cross-domain discrepancy by using the attention mechanism after multiple local discriminators. Additionally, we also consider the domain impacts of instance-wise features and backgrounds in images with large domain divergence, a non-negligible factor for improving the domain adaptive detection model performance. Extensive experiments of various domain shift scenarios show that our method exceeds the state-of-the-art results on several public datasets. Furthermore, qualitative visualization and ablation analyzes can demonstrate the validity of our approach for attending the interested regions and instances on domain adaptation.

Details DOI

AAAI Conference 2021 Conference Paper

Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Xingchao Liu
Mao Ye
Dengyong Zhou
Qiang Liu

We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a fullprecision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number. Computationally, we construct the multipoint quantization with an efficient greedy selection procedure, and adaptively decides the number of low precision points on each quantized weight vector based on the error of its output. This allows us to achieve higher precision levels for important weights that greatly influence the outputs, yielding an “effect of mixed precision” but without physical mixed precision implementations (which requires specialized hardware accelerators (Wang et al. 2019)). Empirically, our method can be implemented by common operands, bringing almost no memory and computation overhead. We show that our method outperforms a range of state-of-the-art methods on ImageNet classification and it can be generalized to more challenging tasks like PASCAL VOC object detection.

PDF Details

NeurIPS Conference 2020 Conference Paper

Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework

Dinghuai Zhang
Mao Ye
Chengyue Gong
Zhanxing Zhu
Qiang Liu

Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning. However, most existing methods only leverage Gaussian smoothing noise and only work for $\ell_2$ perturbation. We propose a general framework of adversarial certification with non-Gaussian noise and for more general types of attacks, from a unified \functional optimization perspective. Our new framework allows us to identify a key trade-off between accuracy and robustness via designing smoothing distributions, helping to design new families of non-Gaussian smoothing distributions that work more efficiently for different $\ell_p$ settings, including $\ell_1$, $\ell_2$ and $\ell_\infty$ attacks. Our proposed methods achieve better certification results than previous works and provide a new perspective on randomized smoothing certification.

PDF Details

NeurIPS Conference 2020 Conference Paper

Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough

Mao Ye
Lemeng Wu
Qiang Liu

Despite the great success of deep learning, recent works show that large deep neural networks are often highly redundant and can be significantly reduced in size. However, the theoretical question of how much we can prune a neural network given a specified tolerance of accuracy drop is still open. This paper provides one answer to this question by proposing a greedy optimization based pruning method. The proposed method has the guarantee that the discrepancy between the pruned network and the original network decays with exponentially fast rate w. r. t. the size of the pruned network, under weak assumptions that apply for most practical settings. Empirically, our method improves prior arts on pruning various network architectures including ResNet, MobilenetV2/V3 on ImageNet.

PDF Details

NeurIPS Conference 2020 Conference Paper

Stein Self-Repulsive Dynamics: Benefits From Past Samples

Mao Ye
Tongzheng Ren
Qiang Liu

We propose a new Stein self-repulsive dynamics for obtaining diversified samples from intractable un-normalized distributions. Our idea is to introduce Stein variational gradient as a repulsive force to push the samples of Langevin dynamics away from the past trajectories. This simple idea allows us to significantly decrease the auto-correlation in Langevin dynamics and hence increase the effective sample size. Importantly, as we establish in our theoretical analysis, the asymptotic stationary distribution remains correct even with the addition of the repulsive force, thanks to the special properties of the Stein variational gradient. We perform extensive empirical studies of our new algorithm, showing that our method yields much higher sample efficiency and better uncertainty estimation than vanilla Langevin dynamics.

PDF Details

ICML Conference 2018 Conference Paper

Variable Selection via Penalized Neural Network: a Drop-Out-One Loss Approach

Mao Ye
Yan Sun 0011

We propose a variable selection method for high dimensional regression models, which allows for complex, nonlinear, and high-order interactions among variables. The proposed method approximates this complex system using a penalized neural network and selects explanatory variables by measuring their utility in explaining the variance of the response variable. This measurement is based on a novel statistic called Drop-Out-One Loss. The proposed method also allows (overlapping) group variable selection. We prove that the proposed method can select relevant variables and exclude irrelevant variables with probability one as the sample size goes to infinity, which is referred to as the Oracle Property. Experimental results on simulated and real world datasets show the efficiency of our method in terms of variable selection and prediction accuracy.

Details

EAAI Journal 2015 Journal Article

Fast crowd density estimation with convolutional neural networks

Min Fu
Pei Xu
Xudong Li
Qihe Liu
Mao Ye
Ce Zhu

As an effective way for crowd control and management, crowd density estimation is an important research topic in artificial intelligence applications. Since the existing methods are hard to satisfy the accuracy and speed requirements of engineering applications, we propose to estimate crowd density by an optimized convolutional neural network (ConvNet). The contributions are twofold: first, convolutional neural network is first introduced for crowd density estimation. The estimation speed is significantly accelerated by removing some network connections according to the observation of the existence of similar feature maps. Second, a cascade of two ConvNet classifier has been designed, which improves both of the accuracy and speed. The method is tested on three data sets: PETS_2009, a Subway image sequence and a ground truth image sequence. Experiments confirm the good performance of the method on the same data sets compared with the state of the art works.

Details DOI