Peng Zhou

TMLR Journal 2026 Journal Article

MV2MAE: Self-Supervised Video Pre-Training with Motion-Aware Multi-View Masked Autoencoders

• Ketul Shah
• Robert Crandall
• Jie Xu
• Peng Zhou
• Vipin Pillai
• Marian George
• Mayank Bansal
• Rama Chellappa

Videos captured from multiple viewpoints can help in perceiving the 3D structure of the world and benefit computer vision tasks such as action recognition, tracking, etc. In this paper, we present MV2MAE, a method for self-supervised learning from synchronized multi-view videos, built on the masked autoencoder framework. We introduce two key enhancements to better exploit multi-view video data. First, we design a cross-view reconstruction task that leverages a cross-attention-based decoder to reconstruct a target viewpoint video from source view. This helps in effectively injecting geometric information and yielding representations robust to viewpoint changes. Second, we introduce a controllable motion-weighted reconstruction loss which emphasizes dynamic regions and mitigates trivial reconstruction of static backgrounds. This improves temporal modeling and encourages learning more meaningful representations across views. MV2MAE achieves state-of-the-art results on the NTU-60, NTU-120 and ETRI datasets among self-supervised approaches. In the more practical transfer learning setting, it delivers consistent gains of +2.0 -- 8.5% on NUCLA, PKU-MMD-II and ROCOG-v2 datasets, demonstrating the robustness and generalizability of our approach. Code: https://github.com/kshah33/mv2mae

TMLR Journal 2026 Journal Article

SpikingBrain: Spiking Brain-inspired Large Models

• Yuqi Pan
• Yupeng Feng
• JingHao Zhuang
• siyu ding
• Han Xu
• Zehao Liu
• Bohan Sun
• Yuhong Chou

Mainstream Transformer-based large language models (LLMs) face significant efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows linearly. These constraints limit their ability to process long sequences effectively. In addition, building large models on non-NVIDIA computing platforms poses major challenges in achieving stable and efficient training and deployment. To address these issues, we introduce SpikingBrain, a new family of brain-inspired models designed for efficient long-context training and inference. SpikingBrain leverages the MetaX GPU cluster and focuses on three core aspects: (1) Model Architecture: linear and hybrid-linear attention architectures with adaptive spiking neurons; (2) Algorithmic Optimizations: an efficient, conversion-based training pipeline compatible with existing LLMs, along with a dedicated spike coding framework; (3) System Engineering: customized training frameworks, operator libraries, and parallelism strategies tailored to the MetaX hardware. Using these techniques, we develop two models: SpikingBrain-7B, a linear LLM, and SpikingBrain-76B, a hybrid-linear MoE LLM. These models demonstrate the feasibility of large-scale LLM development on non-NVIDIA platforms, and our training framework supports weeks of stable training on hundreds of MetaX GPUs with Model FLOPs Utilization (MFU) at expected levels. SpikingBrain achieves performance comparable to open-source Transformer baselines while using exceptionally low data resources (continual pre-training of approximately 150B tokens). Our models also significantly improve long-context efficiency and deliver inference with (partially) constant memory and event-driven spiking behavior. For example, SpikingBrain-7B achieves more than 100× speedup in Time to First Token (TTFT) for 4M-token sequences. Furthermore, the proposed spiking scheme achieves 69.15% sparsity, enabling low-power operation. Overall, this work demonstrates the potential of brain-inspired mechanisms to drive the next generation of efficient and scalable large model design.

AAAI Conference 2025 Conference Paper

Collaborative Similarity Fusion and Consistency Recovery for Incomplete Multi-view Clustering

• Bingbing Jiang
• Chenglong Zhang
• Xinyan Liang
• Peng Zhou
• Jie Yang
• Xingyu Wu
• Junyi Guan
• Weiping Ding

As partial samples are often absent in certain views, incomplete multi-view clustering has become a challenging task. To tackle data with missing views, current methods either utilize the data similarity relations to recover missing samples or primarily consider the available information of existing samples, typically facing some inherent limitations. Firstly, traditional solutions cannot fully explore the potential information contained in missing samples due to their omission strategy, leading to sub-optimal graphs. Moreover, most methods mainly focus on data recovery from the view level, ignoring the differences among available/missing samples in various views. To this end, we propose a collaborative Similarity Fusion and Consistency Recovery (SFCR) method, which resolves the incomplete multi-view clustering problem by learning a unified similarity graph and recovering missing samples with consistent structures. Specifically, to learn a reliable graph compatible across views, a novel view-to-sample fusion model is designed to adaptively coalesce the view-wise similarities among available samples, not only preserving the complementarity and consistency among views but also properly balancing different samples. Furthermore, the missing samples are effectively recovered under the guidance of the fused similarity graph, so as to maintain the consistent structure of recovered data across views. In this way, the similarity learning and the missing data recovery benefit from each other in a collaborative reinforcement manner. Meanwhile, SFCR can directly obtain the final clustering labels without additional post-processing. Extensive experiments demonstrate the effectiveness and superiority of SFCR.

IJCAI Conference 2025 Conference Paper

Gradient-based Causal Feature Selection

• Zhaolong Ling
• Mengxiang Guo
• Xingyu Wu
• Debo Cheng
• Peng Zhou
• Tianci Li
• Zhangling Duan

Causal feature selection leverages causal discovery techniques to identify critical features associated with a target variable using observational data. Traditional methodologies primarily rely on constraint-based or score-based techniques, which are fraught with limitations. For example, conditional independence tests often yield unreliable results in the presence of noise and complex data generation processes, while the computational complexity of learning directed acyclic graphs increases exponentially with the number of variables involved. In light of recent advancements in deep learning, gradient-based methods have shown promise for global causal discovery. However, significant challenges arise when focusing on the identification of local causal features, particularly in defining the local causal constraint space to achieve both minimality and completeness. To address these issues, we introduce a novel gradient-based causal feature selection method (GCFS) that leverages an AutoEncoder to simultaneously model the target variable alongside other variables, thereby capturing of causal associations within a divide-and-conquer framework. Additionally, our approach incorporates a mask pruning strategy that transforms the search process into the minimization of a non-cyclic local reconstruction loss objective function. This function is then effectively optimized using a gradient-based method to accurately identify the causal features related to the target variable. Experimental results substantiate that GCFS surpasses existing methodologies across both synthetic and real datasets.

IJCAI Conference 2025 Conference Paper

Hybrid Local Causal Discovery

• Zhaolong Ling
• Honghui Peng
• Yiwen Zhang
• Debo Cheng
• Xingyu Wu
• Peng Zhou
• Kui Yu

Local causal discovery aims to identify and distinguish the direct causes and effects of a target variable from observational data. Due to the inherent incompleteness of local information, popular methods from global causal discovery often face new challenges in local causal discovery tasks, such as 1) erroneous symmetry constraint tests and the resulting cascading errors in constraint-based methods, and 2) confusion within score-based approaches caused by local spurious equivalence classes. To address the above issues, we propose a Hybrid Local Causal Discovery algorithm, called HLCD. Specifically, HLCD initially utilizes a constraint-based approach with the OR rule to obtain a candidate skeleton, which is subsequently refined using a score-based method to eliminate redundant structures. Furthermore, during the local causal orientation phase, HLCD distinguishes between V-structures and equivalence classes by comparing local structure scores between the two, thereby avoiding orientation interference caused by local equivalence class ambiguities. Comprehensive experiments on 14 benchmark Bayesian networks and two real datasets validate that the proposed algorithm outperforms the existing local causal discovery methods.

AAAI Conference 2025 Conference Paper

Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulation

• Fangyuan Wang
• Shipeng Lyu
• Peng Zhou
• Anqing Duan
• Guodong Guo
• David Navarro-Alarcon

Enabling humanoid robots to perform long-horizon mobile manipulation planning in real-world environments based on embodied perception and comprehension abilities has been a longstanding challenge. With the recent rise of large language models (LLMs), there has been a notable increase in the development of LLM-based planners. These approaches either utilize human-provided textual representations of the real world or heavily depend on prompt engineering to extract such representations, lacking the capability to quantitatively understand the environment, such as determining the feasibility of manipulating objects. To address these limitations, we present the Instruction-Augmented Long-Horizon Planning (IALP) system, a novel framework that employs LLMs to generate feasible and optimal actions based on real-time sensor feedback, including grounded knowledge of the environment, in a closed-loop interaction. Distinct from prior works, our approach augments user instructions into PDDL problems by leveraging both the abstract reasoning capabilities of LLMs and grounding mechanisms. By conducting various real-world long-horizon tasks, each consisting of seven distinct manipulatory skills, our results demonstrate that the IALP system can efficiently solve these tasks with an average success rate exceeding 80%. Our proposed method can operate as a high-level planner, equipping robots with substantial autonomy in unstructured environments through the utilization of multi-modal sensor inputs.

AAAI Conference 2025 Conference Paper

Local Causal Discovery Without Causal Sufficiency

• Zhaolong Ling
• Jiale Yu
• Yiwen Zhang
• Debo Cheng
• Peng Zhou
• Xingyu Wu
• Bingbing Jiang
• Kui Yu

Local causal discovery is crucial for revealing the causal relationships between specific variables from data. Existing local causal discovery algorithms are designed under the assumption of causal sufficiency, which states that there are no latent common causes for two or more of the observed variables in data. However, the assumption of causal sufficiency is often violated in practice. To address this issue, we first propose the local Maximal Ancestral Graph (MAG), referred to as LocalMAG, to describe the local causal relationships of the target variable in the MAG. Then, we propose a local causal discovery algorithm without the assumption of causal sufficiency, called LatentLCD, to learn the LocalMAG. Specifically, LatentLCD first uses the traditional parents and children discovery algorithm to identify the local causal skeleton that includes latent variables and verifies it theoretically. It then identifies bidirectional edges by determining whether both the target variable and its adjacent variables are colliders, thereby identifying latent variables in the local structure of the target variable. Extensive experiments on synthetic datasets have validated that the proposed LatentLCD algorithm significantly outperforms the state-of-the-art methods.

IJCAI Conference 2025 Conference Paper

Multi-view Clustering via Multi-granularity Ensemble

• Jie Yang
• Wei Chen
• Feng Liu
• Peng Zhou
• Zhongli Wang
• Xinyan Liang
• Bingbing Jiang

Multi-view clustering aims to integrate complementary information from multiple views to improve clustering performance. However, existing ensemble-based methods suffer from information loss due to their reliance on single-granularity labels, limiting the discriminative capability of learned representations. Meanwhile, representation and graph fusion-based approaches face challenges such as explicit view alignment and manual weight tuning, making them less effective for heterogeneous views with varying data distributions. To address these limitations, we propose a novel multi-view clustering framework via Multi-granularity Ensemble (MGE), fully using the multi-granularity information across diverse views for accurate and consistent clustering. Specifically, MGE first modifies the hierarchical clustering and then leverages it on each view (including the fused view) to achieve multi-granularity labels. Moreover, the cross-view and cross-granularity fusion strategy is designed to learn a robust co-association similarity matrix, which effectively preserves the fine-grained and coarse-grained structures of multi-view data and facilitates subsequent clustering. Therefore, MGE can provide a comprehensive representation of local and global patterns within data, eliminating the requirement for view alignment and weight tuning. Experiments demonstrate that MGE consistently outperforms state-of-the-art methods across multiple datasets, validating its effectiveness and superiority in handling heterogeneous views.

TIST Journal 2025 Journal Article

Online Distributed Heterogeneous Streaming Feature Selection

• Peng Zhou
• Huiqi Deng
• Yunyun Zhang
• Zhaolong Ling
• Xindong Wu

Data are exploding in many fields and may exist in the streaming mode. When the generation speed of massive streaming data far exceeds the processing speed of a single node and the generated data need to be processed in real time, traditional centralized learning models are challenging in meeting the efficiency requirements. Therefore, online distributed learning models emerge. As time progresses, features may continuously emerge from various sources in a distributed and heterogeneous fashion. Therefore, we study the problem of online distributed heterogeneous streaming feature selection and propose a novel framework to address it, named DHSFS. The framework comprises two main components: sub-node streaming feature selection and global information synchronization. The sub-node component uses a dynamic strategy to select strong features, discard irrelevant ones, and cache weakly relevant features. In the global information synchronization stage, each sub-node synchronizes statistics information with the master node to adjust the global thresholds dynamically. Finally, the features selected by each sub-node are summarized and output. Experiments on 16 datasets show that the DHSFS framework has both high prediction accuracy and high efficiency of online stream feature selection.

AAAI Conference 2025 Conference Paper

Sharper Error Bounds in Late Fusion Multi-view Clustering with Eigenvalue Proportion Optimization

• Liang Du
• Henghui Jiang
• Xiaodong Li
• Yiqing Guo
• Yan Chen
• Feijiang Li
• Peng Zhou
• Yuhua Qian

Multi-view clustering (MVC) aims to integrate complementary information from multiple views to enhance clustering performance. Late Fusion Multi-View Clustering (LFMVC) has shown promise by synthesizing diverse clustering results into a unified consensus. However, current LFMVC methods struggle with noisy and redundant partitions and often fail to capture high-order correlations across views. To address these limitations, we present a novel theoretical framework for analyzing the generalization error bounds of multiple kernel k-means, leveraging local Rademacher complexity and principal eigenvalue proportions. Our analysis establishes a convergence rate of O(1/n), significantly improving upon the existing rate in the order of O(sqrt(k/n)). Building on this insight, we propose a low-pass graph filtering strategy within a multiple linear K-means framework to mitigate noise and redundancy, further refining the principal eigenvalue proportion and enhancing clustering accuracy. Experimental results on benchmark datasets confirm that our approach outperforms state-of-the-art methods in clustering performance and robustness.

IJCAI Conference 2025 Conference Paper

Wavelet Multi-scale Region-Enhanced Network for Medical Image Segmentation

• Hang Lu
• Liang Du
• Peng Zhou

Medical image segmentation is an important task in medical artificial intelligence. Traditional segmentation methods often suffer from the information loss problem, especially in medical image data which contain many different-scale organs or tissues. To address this problem, we propose a novel medical image segmentation method called Wavelet Multi-scale Region-Enhanced Network (WMREN), which has a UNet structure. In the encoder, we design a bi-branch feature extraction architecture, which simultaneously learns the representations with Haar wavelet transform and the residual blocks. The bi-branch architecture can effectively tackle the information loss problem when extracting features. In the decoder we design an innovative Spatial Adaptive Fusion Module to enhance the regions of interest. As we know, the boundaries of objects play an important role in segmentation. To this end, we also carefully design a Contrast Refinement Enhancement Module to highlight the boundaries of the medical objects. Extensive experiments on several benchmark datasets show that our method outperforms state-of-the-art medical image segmentation methods, demonstrating its effectiveness and superiority. The source code is publicly available at https: //github. com/C101812/WMREN/tree/master.

IJCAI Conference 2024 Conference Paper

Active Deep Multi-view Clustering

• Helin Zhao
• Wei Chen
• Peng Zhou

Deep multi-view clustering has been widely studied. However, since it is an unsupervised task, where no labels are used to guide the training, it is still unreliable especially when handling complicated data. Although deep semi-supervised multi-view clustering can alleviate this problem by using some supervised information, the supervised information is often pregiven or randomly selected. Unfortunately, as we know, the clustering performance highly depends on the quality of the supervised information and most of the semi-supervised methods ignore the supervised information selection. To tackle this problem, in this paper, we propose a novel active deep multi-view clustering method, which can actively select important data for querying human annotations. In this method, we carefully design a fusion module, an active selection module, a supervised module, and an unsupervised module, and integrate them into a unified framework seamlessly. In this framework, we can obtain a more reliable clustering result with as few annotations as possible. The extensive experiments on benchmark data sets show that our method can outperform state-of-the-art unsupervised and semi-supervised methods, demonstrating the effectiveness and superiority of the proposed method. The code is available at https: //github. com/wodedazhuozi/ADMC.

IJCAI Conference 2024 Conference Paper

Efficient Multi-view Unsupervised Feature Selection with Adaptive Structure Learning and Inference

• Chenglong Zhang
• Yang Fang
• Xinyan Liang
• Han Zhang
• Peng Zhou
• Xingyu Wu
• Jie Yang
• Bingbing Jiang

As data with diverse representations become high-dimensional, multi-view unsupervised feature selection has been an important learning paradigm. Generally, existing methods encounter the following challenges: (i) traditional solutions either concatenate different views or introduce extra parameters to weight them, affecting the performance and applicability; (ii) emphasis is typically placed on graph construction, yet disregarding the clustering information of data; (iii) exploring the similarity structure of all samples from the original features is suboptimal and extremely time-consuming. To solve this dilemma, we propose an efficient multi-view unsupervised feature selection (EMUFS) to construct bipartite graphs between samples and anchors. Specifically, a parameter-free manner is devised to collaboratively fuse the membership matrices and graphs to learn the compatible structure information across all views, naturally balancing different views. Moreover, EMUFS leverages the similarity relations of data in the feature subspace induced by l2, 0-norm to dynamically update the graph. Accordingly, the cluster information of anchors can be accurately propagated to samples via the graph structure and further guide feature selection, enhancing the quality of selected features and the computational costs in solution processes. A convergent optimization is developed to solve the formulated problem, and experiments demonstrate the effectiveness and efficiency of EMUFS.

NeurIPS Conference 2024 Conference Paper

Fair Kernel K-Means: from Single Kernel to Multiple Kernel

• Peng Zhou
• Rongwen Li
• Liang Du

Kernel k-means has been widely studied in machine learning. However, existing kernel k-means methods often ignore the \textit{fairness} issue, which may cause discrimination. To address this issue, in this paper, we propose a novel Fair Kernel K-Means (FKKM) framework. In this framework, we first propose a new fairness regularization term that can lead to a fair partition of data. The carefully designed fairness regularization term has a similar form to the kernel k-means which can be seamlessly integrated into the kernel k-means framework. Then, we extend this method to the multiple kernel setting, leading to a Fair Multiple Kernel K-Means (FMKKM) method. We also provide some theoretical analysis of the generalization error bound, and based on this bound we give a strategy to set the hyper-parameter, which makes the proposed methods easy to use. At last, we conduct extensive experiments on both the single kernel and multiple kernel settings to compare the proposed methods with state-of-the-art methods to demonstrate their effectiveness.

AAAI Conference 2024 Conference Paper

FocalDreamer: Text-Driven 3D Editing via Focal-Fusion Assembly

• Yuhan Li
• Yishun Dou
• Yue Shi
• Yu Lei
• Xuanhong Chen
• Yi Zhang
• Peng Zhou
• Bingbing Ni

While text-3D editing has made significant strides in leveraging score distillation sampling, emerging approaches still fall short in delivering separable, precise and consistent outcomes that are vital to content creation. In response, we introduce FocalDreamer, a framework that merges base shape with editable parts according to text prompts for fine-grained editing within desired regions. Specifically, equipped with geometry union and dual-path rendering, FocalDreamer assembles independent 3D parts into a complete object, tailored for convenient instance reuse and part-wise control. We propose geometric focal loss and style consistency regularization, which encourage focal fusion and congruent overall appearance. Furthermore, FocalDreamer generates high-fidelity geometry and PBR textures which are compatible with widely-used graphics engines. Extensive experiments have highlighted the superior editing capabilities of FocalDreamer in both quantitative and qualitative evaluations.

NeurIPS Conference 2024 Conference Paper

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

• Yu Zhang
• Songlin Yang
• Ruijie Zhu
• Yue Zhang
• Leyang Cui
• Yiqiao Wang
• Bolun Wang
• Freda Shi

Linear attention Transformers and their gated variants, celebrated for enabling parallel training and efficient recurrent inference, still fall short in recall-intensive tasks compared to traditional Transformers and demand significant resources for training from scratch. This paper introduces Gated Slot Attention (GSA), which enhances Attention with Bounded-memory-Control (ABC) by incorporating a gating mechanism inspired by Gated Linear Attention (GLA). Essentially, GSA comprises a two-layer GLA linked via $\operatorname{softmax}$, utilizing context-aware memory reading and adaptive forgetting to improve memory capacity while maintaining compact recurrent state size. This design greatly enhances both training and inference efficiency through GLA's hardware-efficient training algorithm and reduced state size. Additionally, retaining the $\operatorname{softmax}$ operation is particularly beneficial in ``finetuning pretrained Transformers to RNNs'' (T2R) settings, reducing the need for extensive training from scratch. Extensive experiments confirm GSA's superior performance in scenarios requiring in-context recall and in T2R settings.

AAAI Conference 2024 Conference Paper

Multi-Region Text-Driven Manipulation of Diffusion Imagery

• Yiming Li
• Peng Zhou
• Jun Sun
• Yi Xu

Text-guided image manipulation has attracted significant attention recently. Prevailing techniques concentrate on image attribute editing for individual objects, however, encountering challenges when it comes to multi-object editing. The main reason is the lack of consistency constraints on the spatial layout. This work presents a multi-region guided image manipulation framework, enabling manipulation through region-level textual prompts. With MultiDiffusion as a baseline, we are dedicated to the automatic generation of a rational multi-object spatial distribution, where disparate regions are fused as a unified entity. To mitigate interference from regional fusion, we employ an off-the-shelf model (CLIP) to impose region-aware spatial guidance on multi-object manipulation. Moreover, when applied to the StableDiffusion, the presence of quality-related yet object-agnostic lengthy words hampers the manipulation. To ensure focus on meaningful object-specific words for efficient guidance and generation, we introduce a keyword selection method. Furthermore, we demonstrate a downstream application of our method for multi-region inversion, which is tailored for manipulating multiple objects in real images. Our approach, compatible with variants of Stable Diffusion models, is readily applicable for manipulating diverse objects in extensive images with high-quality generation, showing superb image control capabilities. Code is available at https://github.com/liyiming09/multi-region-guided-diffusion.

AAAI Conference 2024 Conference Paper

Self-Training Based Few-Shot Node Classification by Knowledge Distillation

• Zongqian Wu
• Yujie Mo
• Peng Zhou
• Shangbo Yuan
• Xiaofeng Zhu

Self-training based few-shot node classification (FSNC) methods have shown excellent performance in real applications, but they cannot make the full use of the information in the base set and are easily affected by the quality of pseudo-labels. To address these issues, this paper proposes a new self-training FSNC method by involving the representation distillation and the pseudo-label distillation. Specifically, the representation distillation includes two knowledge distillation methods (i.e., the local representation distillation and the global representation distillation) to transfer the information in the base set to the novel set. The pseudo-label distillation is designed to conduct knowledge distillation on the pseudo-labels to improve their quality. Experimental results showed that our method achieves supreme performance, compared with state-of-the-art methods. Our code and a comprehensive theoretical version are available at https://github.com/zongqianwu/KD-FSNC.

EAAI Journal 2023 Journal Article

Pre-clustering active learning method for automatic classification of building structures in urban areas

• Peng Zhou
• Tongxin Zhang
• Liwen Zhao
• Yifan Qi
• Yuan Chang
• Lu Bai

Identifying the structures of buildings in urban areas is a prerequisite for robust urban planning and regeneration. Owing to the diverse structural designs of urban buildings, automated approaches are required to classify building structures. Supervised machine learning is usually employed to classify various building characteristics. However, this approach requires significant labeling effort. Therefore, this paper proposes a new pre-clustering active learning method for building structure classification. The proposed method captures the statistical characteristics of samples and enhances the recognition of the most valuable training samples, thereby substantially reducing the labeling workload and improving the efficiency and effectiveness of classification. This method was tested via the classification of 3718 buildings in Beijing, China, into five common structures. The results showed that the proposed method could reduce labeling effort by 60% while achieving a promising 90% F1 score for overall classification performance, thus indicating its effectiveness.

AAAI Conference 2023 Conference Paper

Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning

• Bingchen Huang
• Zhineng Chen
• Peng Zhou
• Jiayin Chen
• Zuxuan Wu

The dynamic expansion architecture is becoming popular in class incremental learning, mainly due to its advantages in alleviating catastrophic forgetting. However, task confu- sion is not well assessed within this framework, e.g., the discrepancy between classes of different tasks is not well learned (i.e., inter-task confusion, ITC), and certain prior- ity is still given to the latest class batch (i.e., old-new con- fusion, ONC). We empirically validate the side effects of the two types of confusion. Meanwhile, a novel solution called Task Correlated Incremental Learning (TCIL) is pro- posed to encourage discriminative and fair feature utilization across tasks. TCIL performs a multi-level knowledge distil- lation to propagate knowledge learned from old tasks to the new one. It establishes information flow paths at both fea- ture and logit levels, enabling the learning to be aware of old classes. Besides, attention mechanism and classifier re- scoring are applied to generate more fair classification scores. We conduct extensive experiments on CIFAR100 and Ima- geNet100 datasets. The results demonstrate that TCIL con- sistently achieves state-of-the-art accuracy. It mitigates both ITC and ONC, while showing advantages in battle with catas- trophic forgetting even no rehearsal memory is reserved. Source code: https://github.com/YellowPancake/TCIL.

IJCAI Conference 2023 Conference Paper

Totally Dynamic Hypergraph Neural Networks

• Peng Zhou
• Zongqian Wu
• Xiangxiang Zeng
• Guoqiu Wen
• Junbo Ma
• Xiaofeng Zhu

Recent dynamic hypergraph neural networks (DHGNNs) are designed to adaptively optimize the hypergraph structure to avoid the dependence on the initial hypergraph structure, thus capturing more hidden information for representation learning. However, most existing DHGNNs cannot adjust the hyperedge number and thus fail to fully explore the underlying hypergraph structure. This paper proposes a new method, namely, totally hypergraph neural network (TDHNN), to adjust the hyperedge number for optimizing the hypergraph structure. Specifically, the proposed method first captures hyperedge feature distribution to obtain dynamical hyperedge features rather than fixed ones, by conducting the sampling from the learned distribution. The hypergraph is then constructed based on the attention coefficients of both sampled hyperedges and nodes. The node features are dynamically updated by designing a simple hypergraph convolution algorithm. Experimental results on real datasets demonstrate the effectiveness of the proposed method, compared to SOTA methods. The source code can be accessed via https: //github. com/HHW-zhou/TDHNN.

IJCAI Conference 2022 Conference Paper

Information Augmentation for Few-shot Node Classification

• Zongqian Wu
• Peng Zhou
• Guoqiu Wen
• Yingying Wan
• Junbo Ma
• Debo Cheng
• Xiaofeng Zhu

Although meta-learning and metric learning have been widely applied for few-shot node classification (FSNC), some limitations still need to be addressed, such as expensive time costs for the meta-train and difficult of exploring the complex structure inherent the graph data. To address in issues, this paper proposes a new data augmentation method to conduct FSNC on the graph data including parameter initialization and parameter fine-tuning. Specifically, parameter initialization only conducts a multi-classification task on the base classes, resulting in good generalization ability and less time cost. Parameter fine-tuning designs two data augmentation methods (i. e. , support augmentation and shot augmentation) on the novel classes to generate sufficient node features so that any traditional supervised classifiers can be used to classify the query set. As a result, the proposed method is the first work of data augmentation for FSNC. Experiment results show the effectiveness and the efficiency of our proposed method, compared to state-of-the-art methods, in terms of different classification tasks.

JBHI Journal 2021 Journal Article

Inter-Patient Classification With Encoded Peripheral Pulse Series and Multi-Task Fusion CNN: Application in Type 2 Diabetes

• Chun Ouyang
• Zhongxue Gan
• Junjie Zhen
• Yuxiang Guan
• Xing Zhu
• Peng Zhou

Diabetes mellitus, a chronic disease associated with elevated accumulation of glucose in the blood, is generally diagnosed through an invasive blood test such as oral glucose tolerance test (OGTT). An effective method is proposed to test type 2 diabetes using peripheral pulse waves, which can be measured fast, simply and inexpensively by a force sensor on the wrist over the radial artery. A self-designed pulse waves collection platform includes a wristband, force sensor, cuff, air tubes, and processing module. A dataset was acquired clinically for more than one year by practitioners. A group of 127 healthy candidates and 85 patients with type 2 diabetes, all between the ages of 45 and 70, underwent assessments in both OGTT and pulse data collection at wrist arteries. After preprocessing, pulse series were encoded as images using the Gramian angular field (GAF), Markov transition field (MTF), and recurrence plots (RPs). A four-layer multi-task fusion convolutional neural network (CNN) was developed for feature recognition, the network was well-trained within 30 minutes based on our server. Compared to single-task CNN, multi-task fusion CNN was proved better in classification accuracy for nine of twelve settings with empirically selected parameters. The results show that the best accuracy reached 90. 6% using an RP with threshold ∈ of 6000, which is competitive to that using state-of-the-art algorithms in diabetes classification.

JBHI Journal 2021 Journal Article

Reducing False Triggering Caused by Irrelevant Mental Activities in Brain-Computer Interface Based on Motor Imagery

• Lujia Zhou
• Xuewen Tao
• Feng He
• Peng Zhou
• Hongzhi Qi

In recent years, the brain-computer interface (BCI) based on motor imagery (MI) has been considered as a potential post-stroke rehabilitation technology. However, the recognition of MI relies on the event-related desynchronization (ERD) feature, which has poor task specificity. Further, there is the problem of false triggering (irrelevant mental activities recognized as the MI of the target limb). In this paper, we discuss the feasibility of reducing the false triggering rate using a novel paradigm, in which the steady-state somatosensory evoked potential (SSSEP) is combined with the MI (MI-SSSEP). Data from the target (right hand MI) and nontarget task (rest) were used to establish the recognition model, and three kinds of interference tasks were used to test the false triggering performance. In the MI-SSSEP paradigm, ERD and SSSEP features modulated by MI could be used for recognition, while in the MI paradigm, only ERD features could be used. The results showed that the false triggering rate of interference tasks with SSSEP features was reduced to 29. 3%, which was far lower than the 55. 5% seen under the MI paradigm with ERD features. Moreover, in the MI-SSSEP paradigm, the recognition rate of the target and nontarget task was also significantly improved. Further analysis showed that the specificity of SSSEP was significantly higher than that of ERD (p <; 0. 05), but the sensitivity was not significantly different. These results indicated that SSSEP modulated by MI could more specifically decode the target task MI, and thereby may have potential in achieving more accurate rehabilitation training.

AAAI Conference 2021 Conference Paper

Tri-level Robust Clustering Ensemble with Multiple Graph Learning

• Peng Zhou
• Liang Du
• Yi-Dong Shen
• Xuejun Li

Clustering ensemble generates a consensus clustering result by integrating multiple weak base clustering results. Although it often provides more robust results compared with single clustering methods, it still suffers from the robustness problem if it does not treat the unreliability of base results carefully. Conventional clustering ensemble methods often use all data for ensemble, while ignoring the noises or outliers on the data. Although some robust clustering ensemble methods are proposed, which extract the noises on the data, they still characterize the robustness in a single level, and thus they cannot comprehensively handle the complicated robustness problem. In this paper, to address this problem, we propose a novel Tri-level Robust Clustering Ensemble (TRCE) method by transforming the clustering ensemble problem to a multiple graph learning problem. Just as its name implies, the proposed method tackles robustness problem in three levels: base clustering level, graph level and instance level. By considering the robustness problem in a more comprehensive way, the proposed TRCE can achieve a more robust consensus clustering result. Experimental results on benchmark datasets also demonstrate it. Our method often outperforms other state-ofthe-art clustering ensemble methods. Even compared with the robust ensemble methods, ours also performs better.

AAAI Conference 2020 Conference Paper

Generate, Segment, and Refine: Towards Generic Manipulation Segmentation

• Peng Zhou
• Bor-Chun Chen
• Xintong Han
• Mahyar Najibi
• Abhinav Shrivastava
• Ser-Nam Lim
• Larry Davis

Detecting manipulated images has become a significant emerging challenge. The advent of image sharing platforms and the easy availability of advanced photo editing software have resulted in a large quantities of manipulated images being shared on the internet. While the intent behind such manipulations varies widely, concerns on the spread of false news and misinformation is growing. Current state of the art methods for detecting these manipulated images suffers from the lack of training data due to the laborious labeling process. We address this problem in this paper, for which we introduce a manipulated image generation process that creates true positives using currently available datasets. Drawing from traditional work on image blending, we propose a novel generator for creating such examples. In addition, we also propose to further create examples that force the algorithm to focus on boundary artifacts during training. Strong experimental results validate our proposal.

AAAI Conference 2020 Conference Paper

K-BERT: Enabling Language Representation with Knowledge Graph

• Weijie Liu
• Peng Zhou
• Zhe Zhao
• Zhiruo Wang
• Qi Ju
• Haotang Deng
• Ping Wang

Pre-trained language representation models, such as BERT, capture a general language representation from large-scale corpora, but lack domain-specific knowledge. When reading a domain text, experts make inferences with relevant knowledge. For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge. However, too much knowledge incorporation may divert the sentence from its correct meaning, which is called knowledge noise (KN) issue. To overcome KN, K-BERT introduces softposition and visible matrix to limit the impact of knowledge. K-BERT can easily inject domain knowledge into the models by being equipped with a KG without pre-training by itself because it is capable of loading model parameters from the pre-trained BERT. Our investigation reveals promising results in twelve NLP tasks. Especially in domain-specific tasks (including finance, law, and medicine), K-BERT significantly outperforms BERT, which demonstrates that K-BERT is an excellent choice for solving the knowledge-driven problems that require experts.

AAAI Conference 2020 Conference Paper

Revisiting Image Aesthetic Assessment via Self-Supervised Feature Learning

• Kekai Sheng
• Weiming Dong
• Menglei Chai
• Guohui Wang
• Peng Zhou
• Feiyue Huang
• Bao-Gang Hu
• Rongrong Ji

Visual aesthetic assessment has been an active research field for decades. Although latest methods have achieved promising performance on benchmark datasets, they typically rely on a large number of manual annotations including both aesthetic labels and related image attributes. In this paper, we revisit the problem of image aesthetic assessment from the selfsupervised feature learning perspective. Our motivation is that a suitable feature representation for image aesthetic assessment should be able to distinguish different expertdesigned image manipulations, which have close relationships with negative aesthetic effects. To this end, we design two novel pretext tasks to identify the types and parameters of editing operations applied to synthetic instances. The features from our pretext tasks are then adapted for a one-layer linear classifier to evaluate the performance in terms of binary aesthetic classification. We conduct extensive quantitative experiments on three benchmark datasets and demonstrate that our approach can faithfully extract aesthetics-aware features and outperform alternative pretext schemes. Moreover, we achieve comparable results to state-of-the-art supervised methods that use 10 million labels from ImageNet.

IJCAI Conference 2020 Conference Paper

Self-paced Consensus Clustering with Bipartite Graph

• Peng Zhou
• Liang Du
• Xuejun Li

Consensus clustering provides a framework to ensemble multiple clustering results to obtain a consensus and robust result. Most existing consensus clustering methods usually apply all data to ensemble learning, whereas ignoring the side effects caused by some difficult or unreliable instances. To tackle this problem, we propose a novel self-paced consensus clustering method to gradually involve instances from more reliable to less reliable ones into the ensemble learning. We first construct an initial bipartite graph from the multiple base clustering results, where the nodes represent the instances and clusters and the edges indicate that an instance belongs to a cluster. Then, we learn a structured bipartite graph from the initial one by self-paced learning, i. e. , we automatically decide the reliability of each edge and involves the edges into graph learning in order of their reliability. At last, we obtain the final consensus clustering result from the learned bipartite graph. The extensive experimental results demonstrate the effectiveness and superiority of the proposed method.

IJCAI Conference 2018 Conference Paper

Live Face Verification with Multiple Instantialized Local Homographic Parameterization

• Chen Lin
• Zhouyingcheng Liao
• Peng Zhou
• Jianguo Hu
• Bingbing Ni

State-of-the-art live face verification methods would easily be attacked by recorded facial expression sequence. This work directly addresses this issue via proposing a patch-wise motion parameterization based verification network infrastructure. This method directly explores the underlying subtle motion difference between the facial movements re-captured from a planer screen (e. g. , a pad) and those from a real face; therefore interactive facial expression is no longer required. Furthermore, inspired by the fact that? a fake facial movement sequence MUST contains many patch-wise fake sequences? , we embed our network into a multiple instance learning framework, which further enhance the recall rate of the proposed technique. Extensive experimental results on several face benchmarks well demonstrate the superior performance of our method.

YNIMG Journal 2016 Journal Article

Use of a steady-state baseline to address evoked vs. oscillation models of visual evoked potential origin

• Minpeng Xu
• Yihong Jia
• Hongzhi Qi
• Yong Hu
• Feng He
• Xin Zhao
• Peng Zhou
• Lixin Zhang

There has been a long debate about the neural mechanism of event-related potentials (ERPs). Previously, no evidence or method was apparent to validate the two competing models, the evoked model and the oscillation model. One argument is whether the pre-stimulus brain oscillation could influence the following ERP. This study carried out an innovative visual oddball task experiment to investigate the dynamic process of visual evoked potentials. A period of stable oscillations of specified dominant frequencies and initial phases, i. e. the steady-state baseline, would be induced before responses to transient stimuli of different contrasts, which could overcome the artifact problem caused by the ‘sorting’ method. The result first revealed a ‘three-period-transition’ for the generation of visual evoked potentials by an objective decomposition. The ERP almost retained the preceding oscillation during the first period, provided an unstable negative potential in the second period, and generated the N1 component in the third period. The cross term analysis showed that the evoked model couldn't be the whole explanation for the ERP generation. Furthermore, the component analysis revealed that the N1 latency was sensitive to the initial phase under the low stimulus contrast (supporting the oscillation model) but not under the high stimulus contrast (supporting the evoked model). It demonstrated that the external stimulus contrast is a significant factor deciding the explicit model for ERPs. Our method and preliminary results may help reconcile the previous, seemly contradictory findings on the ERP mechanism.

AAAI Conference 2015 Conference Paper

Convex Batch Mode Active Sampling via α-Relative Pearson Divergence

• Hanmo Wang
• Liang Du
• Peng Zhou
• Lei Shi
• Yi-Dong Shen

Active learning is a machine learning technique that trains a classifier after selecting a subset from an unlabeled dataset for labeling and using the selected data for training. Recently, batch mode active learning, which selects a batch of samples to label in parallel, has attracted a lot of attention. Its challenge lies in the choice of criteria used for guiding the search of the optimal batch. In this paper, we propose a novel approach to selecting the optimal batch of queries by minimizing the α-relative Pearson divergence (RPE) between the labeled and the original datasets. This particular divergence is chosen since it can distinguish the optimal batch more easily than other measures especially when available candidates are similar. The proposed objective is a min-max optimization problem, and it is difficult to solve due to the involvement of both minimization and maximization. We find that the objective has an equivalent convex form, and thus a global optimal solution can be obtained. Then the subgradient method can be applied to solve the simplified convex problem. Our empirical studies on UCI datasets demonstrate the effectiveness of the proposed approach compared with the state-of-the-art batch mode active learning methods.

IJCAI Conference 2015 Conference Paper

Learning a Robust Consensus Matrix for Clustering Ensemble via Kullback-Leibler Divergence Minimization

• Peng Zhou
• Liang Du
• Hanmo Wang
• Lei Shi
• Yi-Dong Shen

Clustering ensemble has emerged as an important extension of the classical clustering problem. It provides a framework for combining multiple base clusterings of a data set to generate a final consensus result. Most existing clustering methods simply combine clustering results without taking into account the noises, which may degrade the clustering performance. In this paper, we propose a novel robust clustering ensemble method. To improve the robustness, we capture the sparse and symmetric errors and integrate them into our robust and consensus framework to learn a low-rank matrix. Since the optimization of the objective function is difficult to solve, we develop a block coordinate descent algorithm which is theoretically guaranteed to converge. Experimental results on real world data sets demonstrate the effectiveness of our method.

IJCAI Conference 2015 Conference Paper

Recovery of Corrupted Multiple Kernels for Clustering

• Peng Zhou
• Liang Du
• Lei Shi
• Hanmo Wang
• Yi-Dong Shen

Kernel-based methods, such as kernel k-means and kernel PCA, have been widely used in machine learning tasks. The performance of these methods critically depends on the selection of kernel functions; however, the challenge is that we usually do not know what kind of kernels is suitable for the given data and task in advance; this leads to research on multiple kernel learning, i. e. we learn a consensus kernel from multiple candidate kernels. Existing multiple kernel learning methods have difficulty in dealing with noises. In this paper, we propose a novel method for learning a robust yet lowrank kernel for clustering tasks. We observe that the noises of each kernel have specific structures, so we can make full use of them to clean multiple input kernels and then aggregate them into a robust, low-rank consensus kernel. The underlying optimization problem is hard to solve and we will show that it can be solved via alternating minimization, whose convergence is theoretically guaranteed. Experimental results on several benchmark data sets further demonstrate the effectiveness of our method.

IJCAI Conference 2015 Conference Paper

Robust Multiple Kernel K-means Using L21-Norm

• Liang Du
• Peng Zhou
• Lei Shi
• Hanmo Wang
• Mingyu Fan
• Wenjian Wang
• Yi-Dong Shen

The k-means algorithm is one of the most often used method for data clustering. However, the standard k-means can only be applied in the original feature space. The kernel k-means, which extends k-means into the kernel space, can be used to capture the non-linear structure and identify arbitrarily shaped clusters. Since both the standard k-means and kernel k-means apply the squared error to measure the distances between data points and cluster centers, a few outliers will cause large errors and dominate the objection function. Besides, the performance of kernel method is largely determined by the choice of kernel. Unfortunately, the most suitable kernel for a particular task is often unknown in advance. In this paper, we first present a robust kmeans using `2, 1-norm in the feature space and then extend it to the kernel space. To recap the powerfulness of kernel methods, we further propose a novel robust multiple kernel k-means (RMKKM) algorithm that simultaneously finds the best clustering label, the cluster membership and the optimal combination of multiple kernels. An alternating iterative schema is developed to find the optimal value. Extensive experiments well demonstrate the effectiveness of the proposed algorithms.

AAMAS Conference 2010 Conference Paper

Using Training Regimens to Teach Expanding Function Approximators

• Peng Zang
• Arya Irani
• Peng Zhou
• Andrea Thomaz
• Charles Isbell

In complex real-world environments, traditional (tabular)techniques for solving Reinforcement Learning (RL) do notscale. Function approximation is needed, but unfortunately, existing approaches generally have poor convergence and optimality guarantees. Additionally, for the case of humanenvironments, it is valuable to be able to leverage humaninput. In this paper we introduce Expanding Value Function Approximation (EVFA), a function approximation algorithm that returns the optimal value function given sufficient rounds. To leverage human input, we introduce a newhuman-agent interaction scheme, training regimens, whichallow humans to interact with and improve agent learning inthe setting of a machine learning game. In experiments, weshow EVFA compares favorably to standard value approximation approaches. We also show that training regimensenable humans to further improve EVFA performance. Inour user study, we find that non-experts are able to provideeffective regimens and that they found the game fun.

ICML Conference 2009 Conference Paper

Discovering options from example trajectories

• Peng Zang
• Peng Zhou
• David Minnen
• Charles Isbell