Author name cluster

Liang Du

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

AAAI Conference 2026 Conference Paper

EvoFMVC: Trusted Federated Multi-View Clustering with Evolutionary Fusion

Li Zhang
Pinhan Fu
Li Lv
Qian Guo
Liang Du
Xinyan Liang

With the growing demand for decentralized collaborative analysis of privacy-sensitive data, federated multi-view clustering (FMVC) has attracted widespread attention due to its ability to balance privacy protection and collaborative modeling. However, current methods still face the following challenges: (1) Clients need to frequently upload high-dimensional data such as model parameters or graph structures, resulting in high communication costs; (2) The structured data uploaded often contains semantic features and has a high risk of being inverted; (3) The server usually merges the data from all clients with the fixed fusion rule, which may result in a suboptimized clustering result when there exist low-quality clients. To address the issues, we propose a new trusted federated multi-view clustering framework (EvoFMVC) that introduces three key innovations: First, lightweight trusted evidence serves as a compact communication medium, significantly reducing overhead compared to conventional model parameters or graph structures. Second, trusted evidences express clustering results in the form of probability distribution, which avoids the risk of structured information being easily inverted. Lastly, we formalize the server-side aggregation process as a neural architecture search (NAS) task where the server flexibly uses different fusion operators to filter and fuse necessary views through evolutionary algorithms, which significantly improves the fusion effect and model performance. Experimental results on multiple datasets show that our method is superior to existing FMVC methods in terms of clustering accuracy and communication efficiency.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Uncertainty-Guided View-Strength-Aware Feature Utilization for Multi-View Classification

Li Lv
Qian Guo
Li Zhang
Liang Du
Bingbing Jiang
Lu Chen
Xinyan Liang

In multi-view classification tasks (MVC), each view provides an unique perspective on the data, offering complementary information that can improve classification performance when properly integrated. However, traditional methods typically adopt a uniform processing strategy for all views before fusion, overlooking the fact that different views may require different treatments due to variations in their quality and informativeness. To address this limitation, we propose a novel framework called Uncertainty-Guided View-Strength-Aware Feature Utilization (UVF) for multi-view classification. Our approach introduces a view uncertainty estimation module to quantify the discriminative strength of each view. Based on this estimation, a Differentiated Feature Selector (DFS) adaptively selects features, retaining informative dimensions in weak views while preserving original features in strong views. Furthermore, we employ an uncertainty-guided fusion strategy that assigns dynamic weights to each view's contribution based on its uncertainty score, enhancing the robustness and reliability of the final decision. Experimental results on benchmark datasets demonstrate that our method significantly outperforms conventional approaches, achieving better classification accuracy and interpretability through strength-aware feature processing and fusion.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

An Association-based Fusion Method for Speech Enhancement

Shijie Wang
Qian Guo
Lu Chen
Liang Du
Zikun Jin
Zhian Yuan
Xinyan Liang

Deep learning-based speech enhancement (SE) methods predominantly draw upon two architectural frameworks: generative adversarial networks and diffusion models. In the realm of SE, capturing the local and global relations between signal frames is crucial for the success of these methods. These frameworks typically employ a UNet architecture as their foundational backbone, integrating Long Short-Term Memory (LSTM) networks or attention mechanisms within the UNet to effectively model both local and global signal relations. However, the coupled relation modeling way may not fully harness the potential of these relations. In this paper, we propose an innovative Association-based Fusion Speech Enhancement method (AFSE), a decoupled method. AFSE first constructs a graph that encapsulates the association between each time window of the speech signal, and then models the global relations between frames by fusing the features of these time windows in a manner akin to graph neural networks. Furthermore, AFSE leverages a UNet with dilated convolutions to model the local relations, enabling the network to maintain a high-resolution representation while benefiting from a wider receptive field. Experimental results demonstrate that the AFSE method significantly improves performance in speech enhancement tasks, validating the effectiveness and superiority of our approach. The code is available at https: //github. com/jie019/AFSE_IJCAI2025.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Dynamic Anchor-based Ensemble Clustering via Hypergraph Reconstruction

Jiaxuan Xu
Lei Duan
Xinye Wang
Liang Du

Ensemble clustering learns a consensus result by integrating a set of base clustering results. Recently, anchor-based methods construct an anchor similarity matrix to represent the affinity relationships among samples, significantly improving computational efficiency. However, these methods struggle with fixed anchors generated by static anchor learning strategies, which lead to low-quality anchor similarity matrix and poor clustering accuracy. To address this issue, we propose a novel method named dynamic anchor-based ensemble clustering via hypergraph reconstruction (YACHT). Specifically, YACHT first transforms the base clustering results into a hypergraph and designs a novel hypergraph enhancement strategy to improve the reliability of the initial hypergraph. YACHT reconstructs the hypergraph through matrix factorization and introduces a mapping matrix to filter out redundant information, capturing a high-quality anchor similarity matrix. Then, YACHT attempts to incorporate the hypergraph into the optimization objective to achieve hypergraph updates. To ensure the accuracy of hypergraph updates, we impose a hypergraph regularizer and a local consensus information alignment term. The alignment term is implemented by minimizing the discrepancy between the label partition derived from the hypergraph regularizer and the local consensus information indicator matrix extracted from the base clustering results. Extensive experimental results demonstrate the outstanding performance of the proposed YACHT. The code is available at https: //github. com/scu-kdde/YACHT.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Improving Evolutionary Multi-View Classification via Eliminating Individual Fitness Bias

Xinyan Liang
Shuai Li
Qian Guo
Yuhua Qian
Bingbing Jiang
Tingjin Luo
Liang Du

Evolutionary multi-view classification (EMVC) methods have gained wide recognition due to their adaptive mechanisms. Fitness evaluation (FE), which aims to calculate the classification performance of each individual in the population and provide reliable performance ranking for subsequent operations, is a core step in such methods. Its accuracy directly determines the correctness of the evolutionary direction. That is, when FE fails to correctly reflect the superiority-inferiority relationship among individuals, it will lead to confusion in individual performance ranking, which in turn misleads the evolutionary direction and results in trapping into local optima. This paper is the first to identify the aforementioned issue in the field of EMVC and call it as fitness evaluation bias (FEB). FEB may be caused by a variety of factors, and this paper approaches the issue from the perspective of view information content: existing methods generally adopt joint training strategies, which restrict the exploration of key information in views with low information content. This makes it difficult for multi-view model (MVM) to achieve optimal performance during convergence, which in turn leads to FE failing to accurately reflect individual performance rankings and ultimately triggering FEB. To address this issue, we propose an evolutionary multi-view classification via eliminating individual fitness bias (EFB-EMVC) method, which alleviates the FEB issue by introducing evolutionary navigators for each MVM, thereby providing more accurate individual ranking. Experimental results fully verify the effectiveness of the proposed method in alleviating the FEB problem, and the EMVC method equipped with this strategy exhibits more superior performance compared with the original EMVC method. (The code is available at https: //github. com/LiShuailzn/Neurips-2025-EFB-EMVC)

PDF Details

AAAI Conference 2025 Conference Paper

k-HyperEdge Medoids for Clustering Ensemble

Feijiang Li
Jieting Wang
Liuya Zhang
Yuhua Qian
Shuai Jin
Tao Yan
Liang Du

Clustering ensemble has been a popular research topic in data science due to its ability to improve the robustness of the single clustering method. Many clustering ensemble methods have been proposed, most of which can be categorized into clustering-view and sample-view methods. The clustering-view method is generally efficient, but it could be affected by the unreliability that existed in base clustering results. The sample-view method shows good performance, while the construction of the pairwise sample relation is time-consuming. In this paper, the clustering ensemble is formulated as a k-HyperEdge Medoids discovery problem and a clustering ensemble method based on k-HyperEdge Medoids that considers the characteristics of the above two types of clustering ensemble methods is proposed. In the method, a set of hyperedges is selected from the clustering view efficiently, then the hyperedges are diffused and adjusted from the sample view guided by a hyperedge loss function to construct an effective k-HyperEdge Medoid set. The loss function is mainly reduced by assigning samples to the hyperedge with the highest degree of belonging. Theoretical analyses show that the solution can approximate the optimal, the assignment method can gradually reduce the loss function, and the estimation of the belonging degree is statistically reasonable. Experiments on artificial data show the working mechanism of the proposed method. The convergence of the method is verified by experimental analysis of twenty data sets. The effectiveness and efficiency of the proposed method are also verified on these data, with nine representative clustering ensemble algorithms as reference.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Sharper Error Bounds in Late Fusion Multi-view Clustering with Eigenvalue Proportion Optimization

Liang Du
Henghui Jiang
Xiaodong Li
Yiqing Guo
Yan Chen
Feijiang Li
Peng Zhou
Yuhua Qian

Multi-view clustering (MVC) aims to integrate complementary information from multiple views to enhance clustering performance. Late Fusion Multi-View Clustering (LFMVC) has shown promise by synthesizing diverse clustering results into a unified consensus. However, current LFMVC methods struggle with noisy and redundant partitions and often fail to capture high-order correlations across views. To address these limitations, we present a novel theoretical framework for analyzing the generalization error bounds of multiple kernel k-means, leveraging local Rademacher complexity and principal eigenvalue proportions. Our analysis establishes a convergence rate of O(1/n), significantly improving upon the existing rate in the order of O(sqrt(k/n)). Building on this insight, we propose a low-pass graph filtering strategy within a multiple linear K-means framework to mitigate noise and redundancy, further refining the principal eigenvalue proportion and enhancing clustering accuracy. Experimental results on benchmark datasets confirm that our approach outperforms state-of-the-art methods in clustering performance and robustness.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

View-Association-Guided Dynamic Multi-View Classification

Xinyan Liang
Li Lv
Qian Guo
Bingbing Jiang
Feijiang Li
Liang Du
Lu Chen

In multi-view classification tasks, integrating information from multiple views effectively is crucial for improving model performance. However, most existing methods fail to fully leverage the complex relationships between views, often treating them independently or using static fusion strategies. In this paper, we propose a View-Association-Guided Dynamic Multi-View Classification method (AssoDMVC) to address these limitations. Our approach dynamically models and incorporates the relationships between different views during the classification process. Specifically, we introduce a view-relation-guided mechanism that captures the dependencies and interactions between views, allowing for more flexible and adaptive feature fusion. This dynamic fusion strategy ensures that each view contributes optimally based on its contextual relevance and the inter-view relationships. Extensive experiments on multiple benchmark datasets demonstrate that our method outperforms traditional multi-view classification techniques, offering a more robust and efficient solution for tasks involving complex multi-view data.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Wavelet Multi-scale Region-Enhanced Network for Medical Image Segmentation

Hang Lu
Liang Du
Peng Zhou

Medical image segmentation is an important task in medical artificial intelligence. Traditional segmentation methods often suffer from the information loss problem, especially in medical image data which contain many different-scale organs or tissues. To address this problem, we propose a novel medical image segmentation method called Wavelet Multi-scale Region-Enhanced Network (WMREN), which has a UNet structure. In the encoder, we design a bi-branch feature extraction architecture, which simultaneously learns the representations with Haar wavelet transform and the residual blocks. The bi-branch architecture can effectively tackle the information loss problem when extracting features. In the decoder we design an innovative Spatial Adaptive Fusion Module to enhance the regions of interest. As we know, the boundaries of objects play an important role in segmentation. To this end, we also carefully design a Contrast Refinement Enhancement Module to highlight the boundaries of the medical objects. Extensive experiments on several benchmark datasets show that our method outperforms state-of-the-art medical image segmentation methods, demonstrating its effectiveness and superiority. The source code is publicly available at https: //github. com/C101812/WMREN/tree/master.

PDF Details DOI

ICLR Conference 2024 Conference Paper

BatchPrompt: Accomplish more with less

Jianzhe Lin
Maurice Diesendruck
Liang Du
Robin Abraham

The ever-increasing token limits of large language models (LLMs) have enabled long context as input. Many LLMs are trained and fine-tuned to perform zero/few-shot inference using instruction-based prompts. Prompts typically include a detailed task instruction, several examples, and a single data point for inference. This baseline is referred to as “SinglePrompt” in this paper. In terms of token count, when the data input is small compared to instructions and examples, this results in lower token utilization, compared with encoder-based models like fine-tuned BERT. This cost inefficiency, affecting inference speed and compute budget, counteracts many of the benefits that LLMs offer. This paper aims to alleviate this problem by batching multiple data points in each prompt, a strategy we refer to as “BatchPrompt”. We improve token utilization by increasing the “density” of data points, however, this cannot be done naively. Simple batching can degrade performance, especially as batch size increases, and data points can yield different answers depending on their position within a prompt. To address the quality issue while retaining high token utilization, we introduce Batch Permutation and Ensembling (BPE) for BatchPrompt – a simple majority vote over repeated permutations of data, that recovers label quality at the cost of more token usage. To counterbalance this cost, we further propose Self-reflection-guided EArly Stopping (SEAS), which can terminate the voting process early for data points that the LLM handles confidently. Our comprehensive experimental evaluation demonstrates that BPE + SEAS can boost the performance of BatchPrompt by a striking margin on a range of popular NLP tasks, including question answering (Boolq), textual entailment (RTE), and duplicate questions identification (QQP). This performance is even competitive with/higher than single-data prompting (SinglePrompt), while using far fewer LLM calls and input tokens. At batch size 32, our BatchPrompt + BPE + SEAS uses 15.7% the number of LLM calls, and achieves: Boolq accuracy 90.6% → 90.9% with 27.4% tokens, QQP accuracy 87.2% → 88.4% with 18.6% tokens, RTE accuracy 91.5% → 91.1% with 30.8% tokens. We hope our simple yet effective approach will shed light on the future research of large language models. Code: github.com/microsoft/BatchPrompt

Details

NeurIPS Conference 2024 Conference Paper

Fair Kernel K-Means: from Single Kernel to Multiple Kernel

Peng Zhou
Rongwen Li
Liang Du

Kernel k-means has been widely studied in machine learning. However, existing kernel k-means methods often ignore the \textit{fairness} issue, which may cause discrimination. To address this issue, in this paper, we propose a novel Fair Kernel K-Means (FKKM) framework. In this framework, we first propose a new fairness regularization term that can lead to a fair partition of data. The carefully designed fairness regularization term has a similar form to the kernel k-means which can be seamlessly integrated into the kernel k-means framework. Then, we extend this method to the multiple kernel setting, leading to a Fair Multiple Kernel K-Means (FMKKM) method. We also provide some theoretical analysis of the generalization error bound, and based on this bound we give a strategy to set the hyper-parameter, which makes the proposed methods easy to use. At last, we conduct extensive experiments on both the single kernel and multiple kernel settings to compare the proposed methods with state-of-the-art methods to demonstrate their effectiveness.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Large Language Model Cascades with Mixture of Thought Representations for Cost-Efficient Reasoning

Murong Yue
Jie Zhao
Min Zhang
Liang Du
Ziyu Yao 0002

Large language models (LLMs) such as GPT-4 have exhibited remarkable performance in a variety of tasks, but this strong performance often comes with the high expense of using paid API services. In this paper, we are motivated to study building an LLM "cascade" to save the cost of using LLMs, particularly for performing (e.g., mathematical, causal) reasoning tasks. Our cascade pipeline follows the intuition that simpler questions can be addressed by a weaker but more affordable LLM, whereas only the most challenging questions necessitate the stronger and more expensive LLM. To realize this decision-making, we consider the "answer consistency" of the weaker LLM as a signal of the question difficulty and propose several methods for answering sampling and consistency checking, including one leveraging a mixture of two thought representations (i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and stronger LLMs, respectively, our cascade pipeline demonstrates comparable performance but reduces about 60% of the cost compared with fully using the stronger LLM.

Details

EAAI Journal 2023 Journal Article

Deep learning-powered vessel traffic flow prediction with spatial-temporal attributes and similarity grouping

Yan Li
Maohan Liang
Huanhuan Li
Zaili Yang
Liang Du
Zhongshuo Chen

Perceiving the future trend of Vessel Traffic Flow (VTF) in advance has great application values in the maritime industry. However, using such big data from the Automatic Identification System (AIS) for accurate VTF prediction remains challenging. Deep training networks can learn valuable features from extensive historical data. This paper proposes a new learning-based prediction network, improved Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) with similarity grouping, including three views. To effectively enable the training network to capture the temporal and periodic (i. e. a spatial attribute) change characteristics of VTF, the CNN and LSTM are employed to compose spatial and temporal views, respectively. Hence, the original one-dimensional data is transformed into a matrix (hour of the day ✕ day) to adapt the input of the proposed methodology. In practical applications, VTF of multiple adjacent target regions need to be predicted simultaneously, and the changes of VTF in different areas may influence each other. To explore their hidden relationships, the similarity grouping view aims to find the target area that exhibits the most similarity with the VTF change trend of the current research area. Furthermore, similar information is combined with the features generated from the other two views to obtain the prediction results. In summary, the new advantage lies in mining the spatiotemporal attributes of data and fusing the similarity information of adjacent regions. Comparative experiments with eleven other methods on realistic VTF datasets show that the proposed method demonstrates superior prediction accuracy and stability performance.

Details DOI

AAAI Conference 2022 Conference Paper

Modify Self-Attention via Skeleton Decomposition for Effective Point Cloud Transformer

Jiayi Han
Longbin Zeng
Liang Du
Xiaoqing Ye
Weiyang Ding
Jianfeng Feng

Although considerable progress has been achieved regarding the transformers in recent years, the large number of parameters, quadratic computational complexity, and memory cost conditioned on long sequences make the transformers hard to train and implement, especially in edge computing configurations. In this case, a dizzying number of works have sought to make improvements around computational and memory efficiency upon the original transformer architecture. Nevertheless, many of them restrict the context in the attention to seek a trade-off between cost and performance with prior knowledge of orderly stored data. It is imperative to dig deep into an efficient feature extractor for point clouds due to their irregularity and a large number of points. In this paper, we propose a novel skeleton decomposition-based self-attention (SD-SA) which has no sequence length limit and exhibits favorable scalability in long-sequence models. Due to the numerical low-rank nature of self-attention, we approximate it by the skeleton decomposition method while maintaining its effectiveness. At this point, we have shown that the proposed method works for the proposed approach on point cloud classification, segmentation, and detection tasks on the Model- Net40, ShapeNet, and KITTI datasets, respectively. Our approach significantly improves the efficiency of the point cloud transformer and exceeds other efficient transformers on point cloud tasks in terms of the speed at comparable performance.

PDF Details

AAAI Conference 2021 Conference Paper

Tri-level Robust Clustering Ensemble with Multiple Graph Learning

Peng Zhou
Liang Du
Yi-Dong Shen
Xuejun Li

Clustering ensemble generates a consensus clustering result by integrating multiple weak base clustering results. Although it often provides more robust results compared with single clustering methods, it still suffers from the robustness problem if it does not treat the unreliability of base results carefully. Conventional clustering ensemble methods often use all data for ensemble, while ignoring the noises or outliers on the data. Although some robust clustering ensemble methods are proposed, which extract the noises on the data, they still characterize the robustness in a single level, and thus they cannot comprehensively handle the complicated robustness problem. In this paper, to address this problem, we propose a novel Tri-level Robust Clustering Ensemble (TRCE) method by transforming the clustering ensemble problem to a multiple graph learning problem. Just as its name implies, the proposed method tackles robustness problem in three levels: base clustering level, graph level and instance level. By considering the robustness problem in a more comprehensive way, the proposed TRCE can achieve a more robust consensus clustering result. Experimental results on benchmark datasets also demonstrate it. Our method often outperforms other state-ofthe-art clustering ensemble methods. Even compared with the robust ensemble methods, ours also performs better.

PDF Details

EAAI Journal 2020 Journal Article

Robust empirical wavelet fuzzy cognitive map for time series forecasting

Ruobin Gao
Liang Du
Kum Fai Yuen

Fuzzy cognitive maps have achieved significant success in time series modeling and forecasting. However, fuzzy cognitive maps still contain weakness to handle the nonstationarity and outliers. We propose a novel time series forecasting model based on fuzzy cognitive maps and empirical wavelet transformation in this paper. The empirical wavelet transformation is applied to decompose the original time series into different levels which capture information of different frequencies. Then, the high-order fuzzy cognitive map is trained to model the relationships among all the sub-series generated and original time series. To enhance the robustness of high-order fuzzy cognitive maps against outliers, a novel learning method based on support vector regression is designed. Finally, we divide the summation of each concept value of the high-order fuzzy cognitive map by two to obtain the numerical predictions. A comprehensive empirical study on eight public time series validates the superiority of proposed model compared with the popular baseline models from the literature.

Details DOI

IJCAI Conference 2020 Conference Paper

Self-paced Consensus Clustering with Bipartite Graph

Peng Zhou
Liang Du
Xuejun Li

Consensus clustering provides a framework to ensemble multiple clustering results to obtain a consensus and robust result. Most existing consensus clustering methods usually apply all data to ensemble learning, whereas ignoring the side effects caused by some difficult or unreliable instances. To tackle this problem, we propose a novel self-paced consensus clustering method to gradually involve instances from more reliable to less reliable ones into the ensemble learning. We first construct an initial bipartite graph from the multiple base clustering results, where the nodes represent the instances and clusters and the edges indicate that an instance belongs to a cluster. Then, we learn a structured bipartite graph from the initial one by self-paced learning, i. e. , we automatically decide the reliability of each edge and involves the edges into graph learning in order of their reliability. At last, we obtain the final consensus clustering result from the learned bipartite graph. The extensive experimental results demonstrate the effectiveness and superiority of the proposed method.

PDF Details DOI

IJCAI Conference 2017 Conference Paper

Top-k Supervise Feature Selection via ADMM for Integer Programming

Mingyu Fan
Xiaojun Chang
Xiaoqin Zhang
Di Wang
Liang Du

Recently, structured sparsity inducing based feature selection has become a hot topic in machine learning and pattern recognition. Most of the sparsity inducing feature selection methods are designed to rank all features by certain criterion and then select the k top ranked features, where k is an integer. However, the k top features are usually not the top k features and therefore maybe a suboptimal result. In this paper, we propose a novel supervised feature selection method to directly identify the top k features. The new method is formulated as a classic regularized least squares regression model with two groups of variables. The problem with respect to one group of the variables turn out to be a 0-1 integer programming, which had been considered very hard to solve. To address this, we utilize an efficient optimization method to solve the integer programming, which first replaces the discrete 0-1 constraints with two continuous constraints and then utilizes the alternating direction method of multipliers to optimize the equivalent problem. The obtained result is the top subset with k features under the proposed criterion rather than the subset of k top features. Experiments have been conducted on benchmark data sets to show the effectiveness of proposed method.

PDF Details

AAAI Conference 2015 Conference Paper

Convex Batch Mode Active Sampling via α-Relative Pearson Divergence

Hanmo Wang
Liang Du
Peng Zhou
Lei Shi
Yi-Dong Shen

Active learning is a machine learning technique that trains a classifier after selecting a subset from an unlabeled dataset for labeling and using the selected data for training. Recently, batch mode active learning, which selects a batch of samples to label in parallel, has attracted a lot of attention. Its challenge lies in the choice of criteria used for guiding the search of the optimal batch. In this paper, we propose a novel approach to selecting the optimal batch of queries by minimizing the α-relative Pearson divergence (RPE) between the labeled and the original datasets. This particular divergence is chosen since it can distinguish the optimal batch more easily than other measures especially when available candidates are similar. The proposed objective is a min-max optimization problem, and it is difficult to solve due to the involvement of both minimization and maximization. We find that the objective has an equivalent convex form, and thus a global optimal solution can be obtained. Then the subgradient method can be applied to solve the simplified convex problem. Our empirical studies on UCI datasets demonstrate the effectiveness of the proposed approach compared with the state-of-the-art batch mode active learning methods.

PDF Details

IJCAI Conference 2015 Conference Paper

Learning a Robust Consensus Matrix for Clustering Ensemble via Kullback-Leibler Divergence Minimization

Peng Zhou
Liang Du
Hanmo Wang
Lei Shi
Yi-Dong Shen

Clustering ensemble has emerged as an important extension of the classical clustering problem. It provides a framework for combining multiple base clusterings of a data set to generate a final consensus result. Most existing clustering methods simply combine clustering results without taking into account the noises, which may degrade the clustering performance. In this paper, we propose a novel robust clustering ensemble method. To improve the robustness, we capture the sparse and symmetric errors and integrate them into our robust and consensus framework to learn a low-rank matrix. Since the optimization of the objective function is difficult to solve, we develop a block coordinate descent algorithm which is theoretically guaranteed to converge. Experimental results on real world data sets demonstrate the effectiveness of our method.

PDF Details

IJCAI Conference 2015 Conference Paper

Recovery of Corrupted Multiple Kernels for Clustering

Peng Zhou
Liang Du
Lei Shi
Hanmo Wang
Yi-Dong Shen

Kernel-based methods, such as kernel k-means and kernel PCA, have been widely used in machine learning tasks. The performance of these methods critically depends on the selection of kernel functions; however, the challenge is that we usually do not know what kind of kernels is suitable for the given data and task in advance; this leads to research on multiple kernel learning, i. e. we learn a consensus kernel from multiple candidate kernels. Existing multiple kernel learning methods have difficulty in dealing with noises. In this paper, we propose a novel method for learning a robust yet lowrank kernel for clustering tasks. We observe that the noises of each kernel have specific structures, so we can make full use of them to clean multiple input kernels and then aggregate them into a robust, low-rank consensus kernel. The underlying optimization problem is hard to solve and we will show that it can be solved via alternating minimization, whose convergence is theoretically guaranteed. Experimental results on several benchmark data sets further demonstrate the effectiveness of our method.

PDF Details

IJCAI Conference 2015 Conference Paper

Robust Multiple Kernel K-means Using L21-Norm

Liang Du
Peng Zhou
Lei Shi
Hanmo Wang
Mingyu Fan
Wenjian Wang
Yi-Dong Shen

The k-means algorithm is one of the most often used method for data clustering. However, the standard k-means can only be applied in the original feature space. The kernel k-means, which extends k-means into the kernel space, can be used to capture the non-linear structure and identify arbitrarily shaped clusters. Since both the standard k-means and kernel k-means apply the squared error to measure the distances between data points and cluster centers, a few outliers will cause large errors and dominate the objection function. Besides, the performance of kernel method is largely determined by the choice of kernel. Unfortunately, the most suitable kernel for a particular task is often unknown in advance. In this paper, we first present a robust kmeans using `2, 1-norm in the feature space and then extend it to the kernel space. To recap the powerfulness of kernel methods, we further propose a novel robust multiple kernel k-means (RMKKM) algorithm that simultaneously finds the best clustering label, the cluster membership and the optimal combination of multiple kernels. An alternating iterative schema is developed to find the optimal value. Extensive experiments well demonstrate the effectiveness of the proposed algorithms.

PDF Details

AAAI Conference 2014 Conference Paper

Exploiting Competition Relationship for Robust Visual Recognition

Liang Du
Haibin Ling

Joint learning of similar tasks has been a popular trend in visual recognition and proven to be beneficial. Between-task similarity often provides useful cues, such as feature sharing, for learning visual classifiers. By contrast, the competition relationship between visual recognition tasks (e. g. , content independent writer identification and handwriting recognition) remains largely under-explored. A key challenge in visual recognition is to select the most discriminating features and remove irrelevant features related to intraclass variations. With the help of auxiliary competing tasks, we can identify such features within a joint learning model exploiting the competition relationship. Motivated by this intuition, we propose a novel way to exploit competition relationship for solving visual recognition problems. Specifically, given a target task and its competing tasks, we jointly model them by a generalized additive regression model with a competition constraint. This constraint effectively discourages choosing of irrelevant features (weak learners) that support the auxiliary competing tasks. We name the proposed algorithm CompBoost. In our study, CompBoost is applied to two visual recognition applications: (1) content-independent writer identification from handwriting scripts by exploiting competing tasks of handwriting recognition, and (2) actor-independent facial expression recognition by exploiting competing tasks of face recognition. In both experiments our approach demonstrates promising performance gains by exploiting the between-task competition.

PDF Details

IJCAI Conference 2013 Conference Paper

Towards Robust Co-Clustering

Liang Du
Yi-Dong Shen

Nonnegative Matrix Tri-factorization (NMTF) and its graph regularized extensions have been widely used for co-clustering task to group data points and features simultaneously. However existing methods are sensitive to noises and outliers which is because of the squared loss function is used to measure the quality of data reconstruction and graph regularization. In this paper, we extend GNMTF by introducing a sparse outlier matrix into the data reconstruction function and applying the `1 norm to measure graph dual regularization errors, which leads to a novel Robust Co-Clustering (RCC) method. Accordingly, RCC is expected to obtain a more faithful approximation to the data recovered from sparse outliers, and achieve robust regularization by reducing the regularization errors of unreliable graphs via `1 norm. To solve the optimization problem of RCC, an alternating iterative algorithm is provided and its convergence is also proved. We also show the connection between the sparse outlier matrix in data reconstruction function and the robust Huber M-estimator. Experimental results on real-world data sets show that our RCC consistently outperforms the other algorithms in terms of clustering performance, which validates the effectiveness and robustness of the proposed approach.

PDF Details DOI