Author name cluster

Qiang Wu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

1 author row

JBHI Journal 2026 Journal Article

A Self-Supervised Diffusion Model With Edge Prior for Unpaired LDCT Denoising

Zhen Zhang
Huizhen Zhang
Shaohua Zheng
Lin Pan
Mingjing Yang
Liqin Huang
Qiang Wu
Zhiyong Zhang

Low-dose computed tomography (LDCT) reduces health risks from radiation exposure but introduces imaging noise and artifacts. While numerous studies have employed deep learning for LDCT image denoising, the field continues to face significant challenges. Recent advancements have seen diffusion models applied to overcome issues of over-smoothness and unstable training inherent in prior deep learning approaches. However, the diffusion models face challenges in direct practical applications due to the extensive sampling steps, significant inference time required, and the need for hard-to-obtain paired data during training. To address these difficulties, this paper introduces a self-supervised diffusion model with edge prior for unpaired LDCT denoising. This method enables denoising within a lower-dimensional space, reducing computational complexity. Our proposed approach enhances denoised image clarity by applying prior edge constraints to compressed encodings; it employs a noise-conditioned encoding strategy to facilitate self-supervised image training, enabling the method to be applicable to unpaired CT data; and it utilizes compressed LDCT encoding as intermediate sampling results during the inference process, thereby accelerating sampling and reducing the time required for inference, making the method more real-time capable. Extensive validation across multiple datasets demonstrates that our method achieves competitive performance against state-of-the-art approaches in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual quality (LPIPS), while maintaining a practically acceptable inference time.

Details DOI

EAAI Journal 2026 Journal Article

Image manipulation localization using multi-noise fusion and learnable compression artifacts

Qiang Wu
Weimin Wei
Fengyong Li
Chuan Qin

Image manipulation localization aims to accurately identify and locate the tampered areas in digital images. Existing deep learning-based methods usually overly rely on single-modal features and cannot effectively fuse multi-source feature information, resulting in poor identification performance and robustness in high-resolution scenarios. To address the aforementioned problem, we propose a new multi-stream fusion image manipulation localization architecture. Firstly, an efficient noise-stream fusion module is designed to integrate multi-source noise features, which can effectively filter out feature redundancy to achieve complementary advantages of noise features. Subsequently, we design a compression artifact recognition module by comprehensively utilizing the frequency feature after Discrete Cosine Transform (DCT) transformation, which can significantly improve the model’s resistance to post-processing by suppressing semantic information in the image. Furthermore, we introduce a cross-modal fusion architecture (CMX) to integrate noise and frequency information under different modes. This architecture allows for mutual correction between different modalities, and thus achieves the effective fusion of multi-modal information. A series of extensive experiments verify that the proposed scheme achieves effective localization capability for multi-scale manipulation regions over different image sets, and outperform most of state-of-the-art schemes in terms of detection accuracy, generalization and robustness.

Details DOI

AAAI Conference 2026 Conference Paper

OTARo: Once Tuning for All Precisions Toward Robust On-Device LLMs

Shaoyuan Chen
Zhixuan Chen
Dawei Yang
Zhihang Yuan
Qiang Wu

Large Language Models (LLMs) fine-tuning techniques not only improve the adaptability to diverse downstream tasks, but also mitigate adverse effects of model quantization. Despite this, conventional quantization suffers from its structural limitation that hinders flexibility during the fine-tuning and deployment stages. Practical on-device tasks demand different quantization precisions (i.e. different bit-widths), e.g., understanding tasks tend to exhibit higher tolerance to reduced precision compared to generation tasks. Conventional quantization, typically relying on scaling factors that are incompatible across bit-widths, fails to support the on-device switching of precisions when confronted with complex real‑world scenarios. To overcome the dilemma, we propose OTARo, a novel method that enables on-device LLMs to flexibly switch quantization precisions while maintaining performance robustness through once fine-tuning. OTARo introduces Shared Exponent Floating Point (SEFP), a distinct quantization mechanism, to produce different bit-widths through simple mantissa truncations of a single model. Moreover, to achieve bit-width robustness in downstream applications, OTARo performs a learning process toward losses induced by different bit-widths. The method involves two critical strategies: (1) Exploitation-Exploration Bit-Width Path Search (BPS), which iteratively updates the search path via a designed scoring mechanism; (2) Low-Precision Asynchronous Accumulation (LAA), which performs asynchronous gradient accumulations and delayed updates under low bit-widths. Experiments on popular LLMs, e.g., LLaMA3.2-1B, LLaMA3-8B, demonstrate that OTARo achieves consistently strong and robust performance for all precisions.

PDF Details DOI

AAAI Conference 2026 Conference Paper

VAEVQ: Enhancing Discrete Visual Tokenization Through Variational Modeling

Sicheng Yang
Xing Hu
Qiang Wu
Dawei Yang

Vector quantization (VQ) transforms continuous image features into discrete representations, providing compressed, tokenized inputs for generative models. However, VQ-based frameworks suffer from several issues, such as non-smooth latent spaces, weak alignment between representations before and after quantization, and poor coherence between the continuous and discrete domains. These issues lead to unstable codeword learning and underutilized codebooks, ultimately degrading the performance of both reconstruction and downstream generation tasks. To this end, we propose VAEVQ, which comprises three key components: (1) Variational Latent Quantization (VLQ), replacing the AE with a VAE for quantization to leverage its structured and smooth latent space, thereby facilitating more effective codeword activation; (2) Representation Coherence Strategy (RCS), adaptively modulating the alignment strength between pre- and post-quantization features to enhance consistency and prevent overfitting to noise; and (3) Distribution Consistency Regularization (DCR), aligning the entire codebook distribution with the continuous latent distribution to improve utilization. Extensive experiments on two benchmark datasets demonstrate that VAEVQ outperforms state-of-the-art methods.

PDF Details DOI

AAAI Conference 2025 Conference Paper

MeRino: Entropy-Driven Design for Generative Language Models on IoT Devices

Youpeng Zhao
Ming Lin
Huadong Tang
Qiang Wu
Jun Wang

Generative Large Language Models (LLMs) stand as a revolutionary advancement in the modern era of artificial intelligence (AI). However, scaling down LLMs for resource-constrained hardware, such as Internet-of-Things (IoT) devices requires non-trivial efforts and domain knowledge. In this paper, we propose a novel information-entropy framework for designing mobile-friendly generative language models. The whole design procedure involves solving a mathematical programming (MP) problem, which can be done on the CPU within minutes, making it nearly zero-cost. We evaluate our designed models, termed MeRino, across fourteen NLP downstream tasks, showing their competitive performance against the state-of-the-art autoregressive transformer models under the mobile setting. Notably, MeRino achieves similar or better performance on both language modeling and zero-shot learning tasks, compared to the 350M parameter OPT while being 4.9x faster on NVIDIA Jetson Nano with 5.5x reduction in model size.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models

Zukang Xu
Xing Hu
Qiang Wu
Dawei Yang

Large language models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their exponentially increasing parameters pose significant challenges for deployment on resource-constrained devices. Vector Quantization (VQ) shows great promise for low-bit quantization (e. g. , 2 to 4 bits), but existing work faces two key challenges: unconstrained direction error and suboptimal bit allocation. In this paper, we propose RSAVQ, a novel VQ framework to enhance extremely low-bit quantization for LLMs. RSAVQ introduces two geometry-driven innovations that effectively mitigate above limitations: (1) Error Direction Sensitivity Guidance (EDSG), which leverages the Fisher information matrix (FIM)-induced Riemannian metric to project quantization errors onto low-sensitivity directions in the parameter space. Specifically, this projection is performed along the negative natural gradient direction, which effectively suppresses error expansion. (2) Weight Channel Sensitivity Guidance (WCSG), which constructs a channel-wise sensitivity metric via FIM curvature analysis to dynamically guide bit resource allocation. The approach facilitates a globally optimal quantization solution within prescribed bit constraints. Experiments demonstrate that RSAVQ outperforms existing methods for LLMs. For example, in 2-bit quantization of LLaMA-3 8B, RSAVQ leads baselines like VPTQ and QuIP# by 0. 4 in perplexity (PPL) and 1. 5 in zero-shot accuracy. This work offers a practical solution for constrained environments and a theoretical bridge between information geometry and the quantization of neural networks, advancing efficient deep learning.

PDF Details

EAAI Journal 2025 Journal Article

Super-resolution reconstruction of sequential images based on an active shift via a hybrid attention calibration mechanism

Qiang Wu
Ziyi Yang
Hongfei Zeng
Jin Zhang
Haojie Xia

Image super-resolution reconstruction converts low-resolution images into high-resolution images, demonstrating extensive potential in processing sequential images. However, most Multi-Image Super-Resolution methods currently face two significant challenges: first, the lack of precision in the shift information between images, as these methods typically rely on algorithms to estimate relative motion. Second, the limited ability to effectively extract subpixel features from low-resolution images directly impacts the richness of details in reconstructed images. This paper proposes a novel active shift-based sequential image super-resolution reconstruction technique to address these issues. This technique integrates hardware control with deep learning algorithms, utilizing a Piezoelectric platform to control camera movement precisely, capturing sequential images with predetermined subpixel shifts, and accurately recording the relative shifts between images. At the algorithmic level, we have designed a hybrid network model that combines a convolutional neural network with a Transformer architecture and integrates channel attention and self-attention mechanisms. This model fully leverages the precise shift information provided by the hardware and significantly enhances the ability to extract image details and overall image quality. Experimental results demonstrate that our method outperforms single-image super-resolution techniques regarding Peak-Signal-to-Noise-Ratio and Structural Similarity Index Measure. To further validate the applicability and effectiveness of this technology, we conducted tests using a resolution test chart, which showed that our technique can increase the resolution of the original imaging system by 25. 6%. Therefore, the strategy combining hardware and software proposed in this paper effectively solves critical issues in Multi-Image Super-Resolution tasks and provides new pathways for image processing technologies.

Details DOI

JBHI Journal 2024 Journal Article

A Siamese-Transport Domain Adaptation Framework for 3D MRI Classification of Gliomas and Alzheimer's Diseases

Luyue Yu
Ju Liu
Qiang Wu
Jing Wang
Aixi Qu

Accurate and fully automated brain structure examination and prediction from 3D volumetric magnetic resonance imaging (MRI) is a necessary step in medical imaging analysis, which can assist greatly in clinical diagnosis. Traditional deep learning models suffer from severe performance degradation when applied to clinically acquired unlabeled data. The performance degradation is mainly caused by domain discrepancy such as different device types and parameter settings for data acquisition. However, existing approaches focus on the reduction of domain discrepancies but ignore the entanglement of semantic features and domain information. In this article, we explore the feature invariance of categories and domains in different projection spaces and propose a Siamese-Transport Domain Adaptation (STDA) method using a joint optimal transport theory and contrastive learning for automatic 3D MRI classification and glioma multi-grade prediction. Specifically, the learning framework updates the distribution of features across domains and categories by Siamese transport network training with an Optimal Cost Transfer Strategy (OCTS) and a Mutual Invariant Constraint (MIC) in two projective spaces to find multiple invariants in potential heterogeneity. We design three sets of transfer task scenarios with different source and target domains, and demonstrate that STDA yields substantially higher generalization performance than other state-of-the-art unsupervised domain adaptation (UDA) methods. The method is applicable on 3D MRI data from glioma to Alzheimer's disease and has promising applications in the future clinical diagnosis and treatment of brain diseases.

Details DOI

AAAI Conference 2024 Conference Paper

Higher-Order Graph Convolutional Network with Flower-Petals Laplacians on Simplicial Complexes

Yiming Huang
Yujie Zeng
Qiang Wu
Linyuan Lü

Despite the recent successes of vanilla Graph Neural Networks (GNNs) on various tasks, their foundation on pairwise networks inherently limits their capacity to discern latent higher-order interactions in complex systems. To bridge this capability gap, we propose a novel approach exploiting the rich mathematical theory of simplicial complexes (SCs) - a robust tool for modeling higher-order interactions. Current SC-based GNNs are burdened by high complexity and rigidity, and quantifying higher-order interaction strengths remains challenging. Innovatively, we present a higher-order Flower-Petals (FP) model, incorporating FP Laplacians into SCs. Further, we introduce a Higher-order Graph Convolutional Network (HiGCN) grounded in FP Laplacians, capable of discerning intrinsic features across varying topological scales. By employing learnable graph filters, a parameter group within each FP Laplacian domain, we can identify diverse patterns where the filters' weights serve as a quantifiable measure of higher-order interaction strengths. The theoretical underpinnings of HiGCN's advanced expressiveness are rigorously demonstrated. Additionally, our empirical investigations reveal that the proposed model accomplishes state-of-the-art performance on a range of graph tasks and provides a scalable and flexible solution to explore higher-order interactions in graphs. Codes and datasets are available at https://github.com/Yiminghh/HiGCN.

PDF Details DOI

EAAI Journal 2024 Journal Article

Instance segmentation algorithm for sorting dismantling components of end-of-life vehicles

Binbin Fan
Xunpeng Qin
Qiang Wu
Jie Fu
Zhongliang Hu
Zhe Wang

In the recycling and dismantling procedures of End-of-Life Vehicles, the diverse thicknesses of disassembled parts necessitate specialized recycling processes, underscoring the imperative for efficient classification and recycling methods. Presently, the classification of disassembled components from scrapped vehicles relies heavily on manual visual inspection and caliper measurements, resulting in inefficiency and low accuracy. To overcome the challenges associated with thickness classification for disassembled components of End-of-Life Vehicles, propose a paradigm shift by introducing an intelligent grading strategy. This approach aims to significantly improve efficiency and accuracy in the classification process, offering a more effective solution for the recycling of dismantled parts. In response to the inefficiencies, low accuracy, and safety concerns associated with manual sorting, propose a deep learning model, incorporating an attention mechanism, for accomplishing instance segmentation tasks of End-of-Life Vehicles components. The model introduces a global attention mechanism into the backbone network for feature extraction, which is further integrated into the instance segmentation branch, resulting in a significant enhancement of instance segmentation performance. The model is trained and optimized using a dataset collected from an End-of-Life Vehicles dismantling facility in China. Experimental results show that when the intersection over union is 0. 5, the mean Average Precision of the model in target detection and instance segmentation reaches 93. 2% and 92. 1% respectively. , and 77. 2% and 72. 4% in the intersection over union is 0. 5–0. 95. In comparison to the unmodified baseline model, the model exhibits improvements of 1. 7%, 2%, 5. 8%, and 3. 1% across evaluation metrics. When contrasted with traditional manual sorting methods, the proposed model exhibits pronounced advantages in terms of accuracy and fairness. In conclusion, the model effectively addresses the challenge of intelligent sorting of dismantled components from End-of-Life Vehicles during the dismantling process.

Details DOI

NeurIPS Conference 2024 Conference Paper

Peri-midFormer: Periodic Pyramid Transformer for Time Series Analysis

Qiang Wu
Gechang Yao
Zhixi Feng
Shuyuan Yang

Time series analysis finds wide applications in fields such as weather forecasting, anomaly detection, and behavior recognition. Previous methods attempted to model temporal variations directly using 1D time series. However, this has been quite challenging due to the discrete nature of data points in time series and the complexity of periodic variation. In terms of periodicity, taking weather and traffic data as an example, there are multi-periodic variations such as yearly, monthly, weekly, and daily, etc. In order to break through the limitations of the previous methods, we decouple the implied complex periodic variations into inclusion and overlap relationships among different level periodic components based on the observation of the multi-periodicity therein and its inclusion relationships. This explicitly represents the naturally occurring pyramid-like properties in time series, where the top level is the original time series and lower levels consist of periodic components with gradually shorter periods, which we call the periodic pyramid. To further extract complex temporal variations, we introduce self-attention mechanism into the periodic pyramid, capturing complex periodic relationships by computing attention between periodic components based on their inclusion, overlap, and adjacency relationships. Our proposed Peri-midFormer demonstrates outstanding performance in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection.

PDF Details DOI

JBHI Journal 2024 Journal Article

TNCB: Tri-Net With Cross-Balanced Pseudo Supervision for Class Imbalanced Medical Image Classification

Aixi Qu
Qiang Wu
Jing Wang
Luyue Yu
Jing Li
Ju Liu

In clinical settings, the implementation of deep neural networks is impeded by the prevalent problems of label scarcity and class imbalance in medical images. To mitigate the need for labeled data, semi-supervised learning (SSL) has gained traction. However, existing SSL schemes exhibit certain limitations. 1) They commonly fail to address the class imbalance problem. Training with imbalanced data makes the model's prediction biased towards majority classes, consequently introducing prediction bias. 2) They usually suffer from training bias arising from unreasonable training strategies, such as strong coupling between the generation and utilization of pseudo labels. To address these problems, we propose a novel SSL framework called Tri-Net with Cross-Balanced pseudo supervision (TNCB). Specifically, two student networks focusing on different learning tasks and a teacher network equipped with an adaptive balancer are designed. This design enables the teacher model to pay more focus on minority classes, thereby reducing prediction bias. Additionally, we propose a virtual optimization strategy to further enhance the teacher model's resistance to class imbalance. Finally, to fully exploit valuable knowledge from unlabeled images, we employ cross-balanced pseudo supervision, where an adaptive cross loss function is introduced to reduce training bias. Extensive evaluation on four datasets with different diseases, image modalities, and imbalance ratios consistently demonstrate the superior performance of TNCB over state-of-the-art SSL methods. These results indicate the effectiveness and robustness of TNCB in addressing imbalanced medical image classification challenges.

Details DOI

JBHI Journal 2023 Journal Article

SDPN: A Slight Dual-Path Network With Local-Global Attention Guided for Medical Image Segmentation

Jing Wang
Shuyi Li
Luyue Yu
Aixi Qu
Qing Wang
Ju Liu
Qiang Wu

Accurate identification of lesions is a key step in surgical planning. However, this task mainly exists two challenges: 1) Due to the complex anatomical shapes of different lesions, most segmentation methods only achieve outstanding performance for a specific structure, rather than other lesions with location differences. 2) The huge number of parameters limits existing transformer-based segmentation models. To overcome these problems, we propose a novel slight dual-path network (SDPN) to segment variable location lesions or organs with significant differences accurately. First, we design a dual-path module to integrate local with global features without obvious memory consumption. Second, a novel Multi-spectrum attention module is proposed to pay further attention to detailed information, which can automatically adapt to the variable segmentation target. Then, the compression module based on tensor ring decomposition is designed to compress convolutional and transformer structures. In the experiment, four datasets, including three benchmark datasets and a clinical dataset, are used to evaluate SDPN. Results of the experiments show that SDPN performs better than other start-of-the-art methods for brain tumor, liver tumor, endometrial tumor and cardiac segmentation. To ensure the generalizability, we train the network on Kvasir-SEG and test on CVC-ClinicDB which collected from a different institution. The quantitative analysis shows that the clinical evaluation results are consistent with the experts. Therefore, this model may be a potential candidate for the segmentation of lesions and organs segmentation with variable locations in clinical applications.

Details DOI

JMLR Journal 2021 Journal Article

Optimal Rates of Distributed Regression with Imperfect Kernels

Hongwei Sun
Qiang Wu

Distributed machine learning systems have been receiving increasing attentions for their efficiency to process large scale data. Many distributed frameworks have been proposed for different machine learning tasks. In this paper, we study the distributed kernel regression via the divide and conquer approach. The learning process consists of three stages. Firstly, the data is partitioned into multiple subsets. Then a base kernel regression algorithm is applied to each subset to learn a local regression model. Finally the local models are averaged to generate the final regression model for the purpose of predictive analytics or statistical inference. This approach has been proved asymptotically minimax optimal if the kernel is perfectly selected so that the true regression function lies in the associated reproducing kernel Hilbert space. However, this is usually, if not always, impractical because kernels that can only be selected via prior knowledge or a tuning process are hardly perfect. Instead it is more common that the kernel is good enough but imperfect in the sense that the true regression can be well approximated by but does not lie exactly in the kernel space. We show distributed kernel regression can still achieve capacity independent optimal rate in this case. To this end, we first establish a general framework that allows to analyze distributed regression with response weighted base algorithms by bounding the error of such algorithms on a single data set, provided that the error bounds have factored the impact of unexplained variance of the response variable. Then we perform a leave one out analysis of the kernel ridge regression and bias corrected kernel ridge regression, which in combination with the aforementioned framework allows us to derive sharp error bounds and capacity independent optimal rates for the associated distributed kernel regression algorithms. As a byproduct of the thorough analysis, we also prove the kernel ridge regression can achieve rates faster than $O(N^{-1})$ (where $N$ is the sample size) in the noise free setting which, to our best knowledge, are first observed and novel in regression learning. [abs] [ pdf ][ bib ] &copy JMLR 2021. ( edit, beta )

PDF Details

AAAI Conference 2021 Conference Paper

PTN: A Poisson Transfer Network for Semi-supervised Few-shot Learning

Huaxi Huang
Junjie Zhang
Jian Zhang
Qiang Wu
Chang Xu

The predicament in semi-supervised few-shot learning (SS- FSL) is to maximize the value of the extra unlabeled data to boost the few-shot learner. In this paper, we propose a Poisson Transfer Network (PTN) to mine the unlabeled information for SSFSL from two aspects. First, the Poisson Merriman–Bence–Osher (MBO) model builds a bridge for the communications between labeled and unlabeled examples. This model serves as a more stable and informative classifier than traditional graph-based SSFSL methods in the message-passing process of the labels. Second, the extra unlabeled samples are employed to transfer the knowledge from base classes to novel classes through contrastive learning. Specifically, we force the augmented positive pairs close while push the negative ones distant. Our contrastive transfer scheme implicitly learns the novel-class embeddings to alleviate the over-fitting problem on the few labeled data. Thus, we can mitigate the degeneration of embedding generality in novel classes. Extensive experiments indicate that PTN outperforms the state-of-the-art few-shot and SSFSL models on miniImageNet and tieredImageNet benchmark datasets.

PDF Details

JMLR Journal 2020 Journal Article

Distributed Minimum Error Entropy Algorithms

Xin Guo
Ting Hu
Qiang Wu

Minimum Error Entropy (MEE) principle is an important approach in Information Theoretical Learning (ITL). It is widely applied and studied in various fields for its robustness to noise. In this paper, we study a reproducing kernel-based distributed MEE algorithm, DMEE, which is designed to work with both fully supervised data and semi-supervised data. The divide-and-conquer approach is employed, so there is no inter-node communication overhead. Similar as other distributed algorithms, DMEE significantly reduces the computational complexity and memory requirement on single computing nodes. With fully supervised data, our proved learning rates equal the minimax optimal learning rates of the classical pointwise kernel-based regressions. Under the semi-supervised learning scenarios, we show that DMEE exploits unlabeled data effectively, in the sense that first, under the settings with weak regularity assumptions, additional unlabeled data significantly improves the learning rates of DMEE. Second, with sufficient unlabeled data, labeled data can be distributed to many more computing nodes, that each node takes only O(1) labels, without spoiling the learning rates in terms of the number of labels. This conclusion overcomes the saturation phenomenon in unlabeled data size. It parallels a recent results for regularized least squares (Lin and Zhou, 2018), and suggests that an inflation of unlabeled data is a solution to the MEE learning problems with decentralized data source for the concerns of privacy protection. Our work refers to pairwise learning and non-convex loss. The theoretical analysis is achieved by distributed U-statistics and error decomposition techniques in integral operators. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

NeurIPS Conference 2020 Conference Paper

Field-wise Learning for Multi-field Categorical Data

Zhibin Li
Jian Zhang
Yongshun Gong
Yazhou Yao
Qiang Wu

We propose a new method for learning with multi-field categorical data. Multi-field categorical data are usually collected over many heterogeneous groups. These groups can reflect in the categories under a field. The existing methods try to learn a universal model that fits all data, which is challenging and inevitably results in learning a complex model. In contrast, we propose a field-wise learning method leveraging the natural structure of data to learn simple yet efficient one-to-one field-focused models with appropriate constraints. In doing this, the models can be fitted to each category and thus can better capture the underlying differences in data. We present a model that utilizes linear models with variance and low-rank constraints, to help it generalize better and reduce the number of parameters. The model is also interpretable in a field-wise manner. As the dimensionality of multi-field categorical data can be very high, the models applied to such data are mostly over-parameterized. Our theoretical analysis can potentially explain the effect of over-parametrization on the generalization of our model. It also supports the variance constraints in the learning objective. The experiment results on two large-scale datasets show the superior performance of our model, the trend of the generalization error bound, and the interpretability of learning outcomes. Our code is available at https: //github. com/lzb5600/Field-wise-Learning.

PDF Details

JMLR Journal 2017 Journal Article

Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network

Zheng-Chu Guo
Lei Shi
Qiang Wu

Distributed learning is an effective way to analyze big data. In distributed regression, a typical approach is to divide the big data into multiple blocks, apply a base regression algorithm on each of them, and then simply average the output functions learnt from these blocks. Since the average process will decrease the variance, not the bias, bias correction is expected to improve the learning performance if the base regression algorithm is a biased one. Regularization kernel network is an effective and widely used method for nonlinear regression analysis. In this paper we will investigate a bias corrected version of regularization kernel network. We derive the error bounds when it is applied to a single data set and when it is applied as a base algorithm in distributed regression. We show that, under certain appropriate conditions, the optimal learning rates can be reached in both situations. [abs] [ pdf ][ bib ] &copy JMLR 2017. ( edit, beta )

PDF Details

JMLR Journal 2010 Journal Article

Learning Gradients: Predictive Models that Infer Geometry and Statistical Dependence

Qiang Wu
Justin Guinney
Mauro Maggioni
Sayan Mukherjee

The problems of dimension reduction and inference of statistical dependence are addressed by the modeling framework of learning gradients. The models we propose hold for Euclidean spaces as well as the manifold setting. The central quantity in this approach is an estimate of the gradient of the regression or classification function. Two quadratic forms are constructed from gradient estimates: the gradient outer product and gradient based diffusion maps. The first quantity can be used for supervised dimension reduction on manifolds as well as inference of a graphical model encoding dependencies that are predictive of a response variable. The second quantity can be used for nonlinear projections that incorporate both the geometric structure of the manifold as well as variation of the response variable on the manifold. We relate the gradient outer product to standard statistical quantities such as covariances and provide a simple and precise comparison of a variety of supervised dimensionality reduction methods. We provide rates of convergence for both inference of informative directions as well as inference of a graphical model of variable dependencies. [abs] [ pdf ][ bib ] &copy JMLR 2010. ( edit, beta )

PDF Details

NeurIPS Conference 2008 Conference Paper

Localized Sliced Inverse Regression

Qiang Wu
Sayan Mukherjee
Feng Liang

We developed localized sliced inverse regression for supervised dimension reduction. It has the advantages of preventing degeneracy, increasing estimation accuracy, and automatic subclass discovery in classification problems. A semisupervised version is proposed for the use of unlabeled data. The utility is illustrated on simulated as well as real data sets.

PDF Details

JMLR Journal 2007 Journal Article

Characterizing the Function Space for Bayesian Kernel Models

Natesh S. Pillai
Qiang Wu
Feng Liang
Sayan Mukherjee
Robert L. Wolpert

Kernel methods have been very popular in the machine learning literature in the last ten years, mainly in the context of Tikhonov regularization algorithms. In this paper we study a coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure. Priors on the random signed measures correspond to prior distributions on the functions mapped by the integral operator. We study several classes of signed measures and their image mapped by the integral operator. In particular, we identify a general class of measures whose image is dense in the reproducing kernel Hilbert space (RKHS) induced by the kernel. A consequence of this result is a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions. We discuss the construction of priors on spaces of signed measures using Gaussian and Lévy processes, with the Dirichlet processes being a special case the latter. Computational issues involved with sampling from the posterior distribution are outlined for a univariate regression and a high dimensional classification problem. [abs] [ pdf ][ bib ] &copy JMLR 2007. ( edit, beta )

PDF Details

NeurIPS Conference 2007 Conference Paper

McRank: Learning to Rank Using Multiple Classification and Gradient Boosting

Ping Li
Qiang Wu
Christopher Burges

We cast the ranking problem as (1) multiple classiﬁcation (“Mc”) (2) multiple or- dinal classiﬁcation, which lead to computationally tractable learning algorithms for relevance ranking in Web search. We consider the DCG criterion (discounted cumulative gain), a standard quality measure in information retrieval. Our ap- proach is motivated by the fact that perfect classiﬁcations result in perfect DCG scores and the DCG errors are bounded by classiﬁcation errors. We propose us- ing the Expected Relevance to convert class probabilities into ranking scores. The class probabilities are learned using a gradient boosting tree algorithm. Evalua- tions on large-scale datasets show that our approach can improve LambdaRank [5] and the regressions-based ranker [6], in terms of the (normalized) DCG scores. An efﬁcient implementation of the boosting tree algorithm is also presented.

PDF Details

JMLR Journal 2006 Journal Article

Estimation of Gradients and Coordinate Covariation in Classification

Sayan Mukherjee
Qiang Wu

We introduce an algorithm that simultaneously estimates a classification function as well as its gradient in the supervised learning framework. The motivation for the algorithm is to find salient variables and estimate how they covary. An efficient implementation with respect to both memory and time is given. The utility of the algorithm is illustrated on simulated data as well as a gene expression data set. An error analysis is given for the convergence of the estimate of the classification function and its gradient to the true classification function and true gradient. [abs] [ pdf ][ bib ] &copy JMLR 2006. ( edit, beta )

PDF Details

JMLR Journal 2004 Journal Article

Support Vector Machine Soft Margin Classifiers: Error Analysis

Di-Rong Chen
Qiang Wu
Yiming Ying
Ding-Xuan Zhou

The purpose of this paper is to provide a PAC error analysis for the q -norm soft margin classifier, a support vector machine classification algorithm. It consists of two parts: regularization error and sample error. While many techniques are available for treating the sample error, much less is known for the regularization error and the corresponding approximation error for reproducing kernel Hilbert spaces. We are mainly concerned about the regularization error. It is estimated for general distributions by a K -functional in weighted L q spaces. For weakly separable distributions (i.e., the margin may be zero) satisfactory convergence rates are provided by means of separating functions. A projection operator is introduced, which leads to better sample error estimates especially for small complexity kernels. The misclassification error is bounded by the V -risk associated with a general class of loss functions V. The difficulty of bounding the offset is overcome. Polynomial kernels and Gaussian kernels are used to demonstrate the main results. The choice of the regularization parameter plays an important role in our analysis. [abs] [ pdf ]

PDF Details