Arrow Research search

Author name cluster

Wei Feng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

55 papers
2 author rows

Possible papers

55

AAAI Conference 2026 Conference Paper

Adversarial Fair Incomplete Multi-View Clustering

  • Qianqian Wang
  • Haiming Xu
  • Wei Feng
  • Quanxue Gao

Fair incomplete multi-view clustering (FIMVC) confronts a critical yet unresolved challenge, as existing methods often fail to address the intertwined issues of data missingness and algorithmic bias simultaneously. In this paper, we propose a novel FIMVC method named Adversarial Fair Incomplete Multi-View Clustering (AFIMVC). The core of AFIMVC is a new adaptive adversarial disentanglement mechanism. This mechanism trains the feature encoder to produce representations that are invariant to sensitive attributes by adversary learning, where the adversarial intensity is dynamically controlled by the model's real-time bias. Additionally, we develop a probabilistic cross-view contrastive learning strategy to achieve semantic consistency in latent space. To handle missing data, AFIMVC employs a context-aware fusion strategy that leverages cross-sample attention to robustly synthesize a unified representation from incomplete views. Extensive experiments demonstrate that AFIMVC achieves a state-of-the-art balance between clustering accuracy and fairness, significantly outperforming existing methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

AutoPP: Towards Automated Product Poster Generation and Optimization

  • Jiahao Fan
  • Yuxin Qin
  • Wei Feng
  • Yanyin Chen
  • Yaoyu Li
  • Ao Ma
  • Yixiu Li
  • Li Zhuang

Product posters blend striking visuals with informative text to highlight the product and capture customer attention. However, crafting appealing posters and manually optimizing them based on online performance is laborious and resource-consuming. To address this, we introduce AutoPP, an automated pipeline for product poster generation and optimization that eliminates the need for human intervention. Specifically, the generator, relying solely on basic product information, first uses a unified design module to integrate the three key elements of a poster (background, text, and layout) into a cohesive output. Then, an element rendering module encodes these elements into condition tokens, efficiently and controllably generating the product poster. Based on the generated poster, the optimizer enhances its Click-Through Rate (CTR) by leveraging online feedback. It systematically replaces elements to gather fine-grained CTR comparisons and utilizes Isolated Direct Preference Optimization (IDPO) to attribute CTR gains to isolated elements. Our work is supported by AutoPP1M, the largest dataset specifically designed for product poster generation and optimization, which contains one million high-quality posters and feedback collected from over one million users. Experiments demonstrate that AutoPP achieves state-of-the-art results in both offline and online settings.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Discriminative Graph Embedding Framework via Label-Free Marginal Fisher Analysis

  • Qianqian Wang
  • Mengping Jiang
  • Wei Feng
  • Haixi Zhang
  • Bin Liu

Marginal Fisher Analysis (MFA) is a classical dimensionality reduction (DR) method that leverages dual graphs to capture intra-class compactness and inter-class separability. However, MFA’s reliance on high-quality labels limits its practical application. For another, existing unsupervised DR methods neglect data’s local manifold relationship, resulting in poor discriminativeness. To address these limitations, we propose a novel DR method named Discriminative Graph Embedding Framework (DGEF) via Label-Free Marginal Fisher Analysis. Our approach uses the adjacency matrix and cluster indicator matrix derived from centerless K-Means to construct intrinsic graph and penalty graph, which preserve the local manifold structure of the data. Additionally, we have derived the convertible relationship between centerless K-Means and Manifold learning and unified them within a graph embedding framework. By adopting the intrinsic graph and penalty graph, our DGEF avoids centroid initialization and ensures robustness and discriminativeness. This method achieves dimensionality reduction adaptively without relying on labeled data. Extensive experiments on benchmark datasets show that our approach outperforms conventional methods in clustering performance.

PDF Details DOI

AAAI Conference 2026 Conference Paper

E-Logic Prompt: Unified Energy-Logic Framework for Continual Visual Question Answering

  • Jiayao Tan
  • Tianle Liu
  • Fuyuan Hu
  • Wei Feng
  • Liang Wan

Prompt tuning has shown promise for continual visual question answering (CVQA), facilitating modular and transferable knowledge across tasks. However, existing approaches often overlook the guiding role of prompts in the model’s implicit reasoning process. This oversight can lead to inconsistent reasoning paths and performance degradation across tasks. To address this issue, we propose the E Logic Prompt framework, which employs energy-based models (EBMs) to model the semantic compatibility between prompts and queries. In this framework, prompts function not only as adapters but also as reasoning guides that help maintain coherence throughout the inference process. The framework enforces logical consistency at three levels. At the input level, it selects semantically aligned prompts by minimizing the energy between queries and prompts. Within the model, it aligns intermediate representations with prompts across layers to preserve step-by-step reasoning. Across tasks, it applies energy-based constraints to regulate prompt behavior, effectively suppressing semantic drift and enabling prompt reuse. These three levels of consistency together enhance the guiding capacity of prompts, allowing them to steer the model toward more stable and coherent reasoning. Extensive experiments show that E Logic Prompt outperforms existing methods in both accuracy and knowledge retention, while effectively maintaining balanced cross-modal reasoning throughout continual learning.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Federated Incomplete Multi-View Clustering with Tensorized Low-Rank Constraint

  • Wei Feng
  • Danting Liu
  • Qianqian Wang
  • Mengping Jiang
  • Bin Liu

Federated Multi-View Clustering has gained increasing attention for its ability to discover complementary clustering structures of distributed multi-view data while preserving data privacy. However, real-world clients often only have access to partial views, and the view incompleteness poses great challenges to federated multi-view feature fusion to exploit consistent and complementary information. Moreover, efficiency is highly expected in federated scenarios due to the limited resources of each client. To alleviate these issues, we propose Federated Incomplete Multi-View Clustering with Tensorized Low-Rank Constraint (FIMVC-TLRC), which incorporates anchors to improve efficiency and is able to address prevalent view incompleteness issue in federated scenarios. FIMVC-TLRC aligns the local anchor graphs and employs a tensorized low-rank constraint based on the tensor Schatten p-norm to enforce the consistency of the data representations learned by each client. Besides, a federated optimization framework is developed to jointly optimize the construction and alignment of anchor graphs, thus enabling collaborative and privacy-preserving training. Experimental results on multiple datasets demonstrate its effectiveness.

PDF Details DOI

AAAI Conference 2026 Conference Paper

MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generation

  • Run Ling
  • Ke Cao
  • Jian Lu
  • Ao Ma
  • Haowei Liu
  • Runze He
  • Changwei Wang
  • Rongtao Xu

Multi-subject video generation aims to synthesize videos from textual prompts and multiple reference images, ensuring that each subject preserves natural scale and visual fidelity. However, current methods face two challenges: scale inconsistency, where variations in subject size lead to unnatural generation, and permutation sensitivity, where the order of reference inputs causes subject distortion. In this paper, we propose MoFu, a unified framework that tackles both challenges. For scale inconsistency, we introduce Scale-Aware Modulation (SMO), an LLM-guided module that extracts implicit scale cues from the prompt and modulates features to ensure consistent subject sizes. To address permutation sensitivity, we present a simple yet effective Fourier Fusion strategy that processes the frequency information of reference features via the Fast Fourier Transform to produce a unified representation. Besides, we design a Scale-Permutation Stability Loss to jointly encourage scale-consistent and permutation-invariant generation. To further evaluate these challenges, we establish a dedicated benchmark with controlled variations in subject scale and reference permutation. Extensive experiments demonstrate that MoFu significantly outperforms existing methods in preserving natural scale, subject fidelity, and overall visual quality.

PDF Details DOI

AAAI Conference 2026 Conference Paper

RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers

  • Ke Cao
  • Jing Wang
  • Ao Ma
  • Jiasong Feng
  • Xuanhua He
  • Run Ling
  • Haowei Liu
  • Jian Lu

The Diffusion Transformer plays a pivotal role in advancing text-to-image and text-to-video generation, owing primarily to its inherent scalability. However, existing controlled diffusion transformer methods incur significant parameter and computational overheads and suffer from inefficient resource allocation due to their failure to account for the varying relevance of control information across different transformer layers. To address this, we propose the Relevance-Guided Efficient Controllable Generation framework, RelaCtrl, enabling efficient and resource-optimized integration of control signals into the Diffusion Transformer. First, we evaluate the relevance of each layer in the Diffusion Transformer to the control information by assessing the ControlNet Relevance Score, which measures the impact of skipping each control layer on both the quality of generation and the control effectiveness during inference. Based on the strength of the relevance, we then tailor the positioning, parameter scale, and modeling capacity of the control layers to reduce unnecessary parameters and redundant computations. Additionally, to further improve efficiency, we replace the self-attention and FFN in the commonly used copy block with the carefully designed Two-Dimensional Shuffle Mixer (TDSM), enabling efficient implementation of both the token mixer and channel mixer. Both qualitative and quantitative experimental results demonstrate that our approach achieves superior performance with only 15% of the parameters and computational complexity compared to PixArt-delta.

PDF Details DOI

AAAI Conference 2026 Conference Paper

UV-RGS: Relightable 3D Gaussian Splatting from Unposed Views Under Varied Illuminations

  • Wei Feng
  • Chi Huang
  • Qi Zhang
  • Qian Zhang
  • Nan Li

The latest advancements in scene relighting have been predominantly driven by inverse rendering with 3D Gaussian Splatting (3DGS). However, existing methods remain overly reliant on precise camera parameters under static illumination conditions, which is prohibitively expensive and even impractical in real-world scenarios. In this paper, we propose a novel learning from Unposed views under Varied illuminations Relightable 3D Gaussian Splatting (dubbed UV-RGS), to address this challenge by jointly optimizing camera poses, 3DGS representations, surface materials, and environment illuminations (i.e., unknown and varied lighting conditions in training) using only unposed views under varied lightings. Firstly, UV-RGS presents a viewpoint dividing strategy to group inputs into constituent units, enabling each unit can perform similar poses and illuminations. Next, for each unit, to get the constituent model, UV-RGS establishes an incrementally pose learning module to estimate coarse camera parameters, which also enjoy a proxy-view refinement to alleviate the sparse view learning. Additionally, for all constituent unit models, we introduce a holistic model learning strategy that integrates progressive unit aggregation component and the 3DGS coupled with camera poses joint optimization, which realizes the scene high-fidelity perception by the physical-based rendering. Extensive experiments on both real-world and synthetic challenging datasets demonstrate the effectiveness of UV-RGS, achieving the state-of-the-art performance for scene inverse rendering by learning 3DGS from only unposed views under varied illuminations.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

2D Gaussian Splatting for Outdoor Scene Decomposition and Relighting

  • Wei Feng
  • Kangrui Ye
  • Qi Zhang
  • Qian Zhang
  • Nan Li

Gaussian splatting techniques have recently revolutionized outdoor scene decomposition and relighting through multi-view images. However, achieving high rendering quality still requires a fixed lighting condition among all input views, which is costly or even impractical to capture in outdoor scenes. In this paper, we propose outdoor scene decomposition and relighting with 2D Gaussian splatting (OSDR-GS), a novel inverse rendering strategy under outdoor changing and unknown lighting conditions. Firstly, we present a lighting-based group learning framework that categorizes input images into multiple lighting groups, to learn the separate lighting from each group individually. Secondly, OSDR-GS introduces a fine-grained outdoor lighting component to represent sun-light and sky-light, respectively, which are also adjusted via the correlative exposure factors adaptively. Finally, we construct a visibility-driven shadow module to characterize the nuanced interplay of light and occlusion realistically, for eliminating the uncertainty of dark pixels on lighting-based group learning. Extensive experiments on multiple challenging outdoor datasets validate the effectiveness of OSDR-GS, which achieves the state-of-the-art performance in changing lighting scene inverse rendering.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Contrastive Multi-view Subspace Clustering via Tensor Transformers Autoencoder

  • Qianqian Wang
  • Zihao Zhang
  • Wei Feng
  • Zhiqiang Tao
  • Quanxue Gao

Multi-view clustering aims to identify consistent and complementary information across multiple views to partition data into clusters, emerging as a popular unsupervised method for multi-view data analysis. However, existing methods often design view-specific encoders to extract distinct features from each view, lacking exploration of their complementarity. Additionally, current contrastive-based multi-view clustering methods may lead to erroneous negative sample pairs conflicting with the clustering objective. To address these challenges, we propose a novel Contrastive Multi-view Subspace Clustering via Tensor Transformers Autoencoder (TTAE). On the one hand, it facilitates information exchange between views by tensor transformers autoencoder, thereby enhancing complementarity. On the other hand, It learns a consistent subspace with a self-expression layer. Meanwhile, adaptive contrastive learning helps to provide more discriminative features for the self-expression learning layer, and the self-expression learning layer in turn supervises contrastive learning. Moreover, our method adaptively selects positive and negative samples for contrastive learning to mitigate the impact of inappropriate negative sample pairs. Extensive experiments on several multi-view datasets demonstrate the effectiveness and superiority of our model.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Deep Multi-modal Graph Clustering via Graph Transformer Network

  • Qianqian Wang
  • Haiming Xu
  • Zihao Zhang
  • Wei Feng
  • Quanxue Gao

Current deep multi-modal graph clustering methods primarily rely on Graph Neural Network (GNN) to fully exploit attribute features and graph structures, including message propagation and low-dimensional feature embedding. However, these methods lack further exploration of graph structural information, such as the relationship between nodes and shortest paths. Additionally, they may not sufficiently mine complementary information among multi-modal graph data. To address these issues, we propose a novel Deep Multi-modal Graph Clustering via Graph Transformer Network method, called DMGC-GTN. This method thoroughly dissects and utilizes graph structural information, applying graph smoothing to node features and incorporating various forms of embeddings into the transformer architecture. This achieves a unified embedding of graph structure and multi-modal feature attributes, fully exploiting the complementary information within multi-modal graph data. Extensive experiments demonstrate the effectiveness of our algorithm.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DOVTrack: Data-Efficient Open-Vocabulary Tracking

  • Zekun Qian
  • Ruize Han
  • Zhixiang Wang
  • Junhui Hou
  • Wei Feng

Open-Vocabulary Multi-Object Tracking (OVMOT) aims to detect and track multi-category objects including both seen and unseen categories during training. Currently, a significant challenge in this domain is the lack of large-scale annotated video data for training. To address this challenge, this work aims to effectively train the OV tracker using only the existing limited and sparsely annotated video data. We propose a comprehensive training sample space expansion strategy that addresses the fundamental limitation of sparse annotations in OVMOT training. Specifically, for the association task, we develop a diffusion-based feature generation framework that synthesizes intermediate object features between sparsely annotated frames, effectively expanding the training sample space by approximately 3× and enabling robust association learning from temporally continuous features. For the detection task, we introduce a dynamic group contrastive learning approach that generates diverse sample groups through affinity, dispersion, and adversarial grouping strategies, tripling the effective training samples for classification while maintaining sample quality. Additionally, we propose an adaptive localization loss that expands positive sample coverage by lowering IoU thresholds while mitigating noise through confidence-based weighting. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the OVMOT benchmark, surpassing existing methods by 3. 8\% in TETA metric, without requiring additional data or annotations. The code will be available at https: //github. com/zekunqian/DOVTrack.

PDF Details

IJCAI Conference 2025 Conference Paper

Enhanced Unsupervised Discriminant Dimensionality Reduction for Nonlinear Data

  • Qianqian Wang
  • Mengping Jiang
  • Wei Feng
  • Zhengming Ding

Linear Discriminant Analysis (LDA) is a classical supervised dimensionality reduction algorithm. However, LDA focuses more on global structure and overly depends on reliable data labels. For data with outliers and nonlinear structures, LDA cannot effectively capture the true structure of the data. Moreover, the subspace dimension learned by LDA must be smaller than cluster number, which limits its practical applications. To address these issues, we propose a novel unsupervised LDA method that combines centerless K-means and LDA. This method eliminates the need to calculate cluster centroids and improves model robustness. By fusing centerless K-means and LDA into a unified framework and deducing the connection between K-means and manifold learning, this method captures the local manifold structure and discriminative structure. Additionally, the dimensionality of the subspace is not restricted. This method not only overcomes the limitations of traditional LDA but also improves the model’s adaptability to complex data. Extensive experiments on seven datasets demonstrate the effectiveness of the proposed method.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Fair Incomplete Multi-View Clustering via Distribution Alignment

  • Qianqian Wang
  • Haiming Xu
  • Meiling Liu
  • Wei Feng
  • Xiangdong Zhang

Incomplete multi-view clustering (IMVC) extracts consistent and complementary information from multi-source/modality data with missing views, aiming to partition the data into different clusters. It can effectively address the problem of unsupervised multi-source data analysis in complex environments and has gained considerable attention. However, the fairness of IMVC remains underexplored, particularly when data contains sensitive features ({e. g. }, gender, marital status, and age). To tackle the problem, this work presents a novel Fair Incomplete Multi-View Clustering (FIMVC) method. The proposed FIMVC introduces fairness constraints to ensure clustering results are independent of sensitive features. Additionally, it learns consensus representations to enhance clustering performance by maximizing mutual information and aligning the distributions of different views. Experimental results on three datasets containing sensitive features demonstrate that our method improves the fairness of clustering results while outperforming state-of-the-art IMVC methods in clustering performance.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Federated Multi-view Graph Clustering with Incomplete Attribute Imputation

  • Wei Feng
  • Zeyu Bi
  • Qianqian Wang
  • Bo Dong

Federated Multi-View Clustering (FedMVC) aims to uncover consistent clustering structures from distributed multi-view data for clustering while preserving data privacy. However, existing FedMVC methods under vertical settings either ignore the ubiquitous incomplete view issue or require uploading data features, which may lead to privacy leakage or induce high communication costs. To mitigate the view incompleteness issue and simultaneously maintain privacy and efffciency, we propose a novel Federated Multiview Graph Clustering with Incomplete Attribute Imputation (FMVC-IAI). This method constructs a consensus graph structure through complementary multi-view data and then utilizes a non-parametric graph neural network (GNN) to impute missing features. Additionally, it utilizes the adjacency graph as the knowledge carrier to share and fuse the multi-view information. To alleviate the high communication cost due to graph sharing, we proposed to share the anchor graph for global adjacency graph construction, which reduces communication cost and also helps to reduce privacy leakage risk. Extensive experiments demonstrate the superiority of our method in FedMVC tasks with incomplete views.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective

  • Wei Feng
  • Zongyuan Ge

Generalized Category Discovery (GCD) aims to leverage labeled samples from known categories to cluster unlabeled data that may include both known and unknown categories. While existing methods have achieved impressive results under standard conditions, their performance often deteriorates in the presence of distribution shifts. In this paper, we explore a more realistic task: Domain-Shifted Generalized Category Discovery (DS_GCD), where the unlabeled data includes not only unknown categories but also samples from unknown domains. To tackle this challenge, we propose a \textbf{\underline{F}}requency-guided Gene\textbf{\underline{r}}alized Cat\textbf{\underline{e}}gory Discov\textbf{\underline{e}}ry framework (FREE) that enhances the model's ability to discover categories under distributional shift by leveraging frequency-domain information. Specifically, we first propose a frequency-based domain separation strategy that partitions samples into known and unknown domains by measuring their amplitude differences. We then propose two types of frequency-domain perturbation strategies: a cross-domain strategy, which adapts to new distributions by exchanging amplitude components across domains, and an intra-domain strategy, which enhances robustness to intra-domain variations within the unknown domain. Furthermore, we extend the self-supervised contrastive objective and semantic clustering loss to better guide the training process. Finally, we introduce a clustering-difficulty-aware resampling technique to adaptively focus on harder-to-cluster categories, further enhancing model performance. Extensive experiments demonstrate that our method effectively mitigates the impact of distributional shifts across various benchmark datasets and achieves superior performance in discovering both known and unknown categories.

PDF Details

IROS Conference 2025 Conference Paper

GeoFlow-SLAM: A Robust Tightly-Coupled RGBD-Inertial and Legged Odometry Fusion SLAM for Dynamic Legged Robotics

  • Tingyang Xiao
  • Xiaolin Zhou
  • Liu Liu
  • Wei Sui
  • Wei Feng
  • Jiaxiong Qiu
  • Xinjie Wang
  • Zhizhong Su

This paper presents GeoFlow-SLAM, a robust and effective Tightly-Coupled RGBD-Inertial and Legged Odometry Fusion SLAM for legged robotics undergoing aggressive and high-frequency motions. By integrating geometric consistency, legged odometry constraints, and dual-stream optical flow (GeoFlow), our method addresses three critical challenges: feature matching and pose initialization failures during fast locomotion and visual feature scarcity in texture-less scenes. Specifically, in rapid motion scenarios, feature matching is notably enhanced by leveraging dual-stream optical flow, which combines prior map points and poses. Additionally, we propose a robust pose initialization method for fast locomotion and IMU error in legged robots, integrating IMU/Legged odometry, inter-frame Perspective-n-Point (PnP), and Generalized Iterative Closest Point (GICP). Furthermore, a novel optimization framework that tightly couples depth-to-map and GICP geometric constraints is first introduced to improve the robustness and accuracy in long-duration, visually texture-less environments. The proposed algorithms achieve state-of-the-art (SOTA) on collected legged robots and open-source datasets. To further promote research and development, the open-source datasets and code will be made publicly available at https://github.com/HorizonRobotics/GeoFlowSlam.

Details

JBHI Journal 2025 Journal Article

Neighbor-Guided Unbiased Framework for Generalized Category Discovery in Medical Image Classification

  • Wei Feng
  • Sijin Zhou
  • Yiwen Jiang
  • Feilong Tang
  • Zongyuan Ge

Generalized category discovery (GCD) utilizes seen category knowledge to automatically discover new semantic categories that are not defined in the training phase. Nevertheless, there has been no research conducted on identifying new classes using medical images and disease categories, which is essential for understanding and diagnosing specific diseases. Moreover, existing methods still produce predictions that are biased towards seen categories since the model is mainly supervised by labeled seen categories, which in turn leads to sub-optimal clustering performance. In this paper, we propose a new neighbor-guided unbiased framework (NGUF) that leverages neighbor information to mitigate prediction bias to address the GCD problem in medical tasks. Specifically, we devise a neighbor-guided cross-pseudo-clustering strategy, which exploits the knowledge of the nearest-neighbor samples to adjust the model predictions thereby generating unbiased pseudo-clustering supervision. Then, based on the unbiased pseudo-clustering supervision, we use a view-invariant learning strategy to assign labels to all samples. In addition, we propose an adaptive weight learning strategy that dynamically determines the degree of adjustment of the predictions of different samples based on the distance density values. Finally, we further propose a cross-batch knowledge distillation module to utilize information from successive iterations to encourage training consistency. Extensive experiments on four medical image datasets show that NGUF is effective in mitigating the model's prediction bias and has superior performance to other state-of-the-art GCD algorithms. Our code will be released soon.

Details DOI

TCS Journal 2025 Journal Article

Packing internally disjoint Steiner paths of modified bubble-sort networks

  • Lina Zhao
  • Shiying Wang
  • Wei Feng

Let G be a connected simple graph with vertex set V ( G ) and edge set E ( G ). For S ⊆ V ( G ) with | S | ≥ 2, a path P in G is said to be an S-Steiner path (or S-path for short) if it connects all vertices of S. Two S-paths P 1 and P 2 are internally disjoint if E ( P 1 ) ∩ E ( P 2 ) = ∅ and V ( P 1 ) ∩ V ( P 2 ) = S. The packing number of internally disjoint S-paths, denoted as π G ( S ), is the maximum number of internally disjoint S-paths in G. For an integer 2 ≤ k ≤ | V ( G ) |, the k-path connectivity of a graph G is defined as π k ( G ) = min { π G ( S ) | S ⊆ V ( G ) and | S | = k }. The modified bubble-sort graph, denoted M B n, is an interconnection network topological model for multiprocessor systems. In this paper, we focus on the 3-path-connectivity of the n-dimensional modified bubble sort graphs. By analyzing the structural properties of M B n, we determine that π 3 ( M B n ) = ⌊ 3 n − 1 4 ⌋ for n ≥ 4.

Details DOI

NeurIPS Conference 2025 Conference Paper

QBasicVSR: Temporal Awareness Adaptation Quantization for Video Super-Resolution

  • Zhenwei Zhang
  • Fanhua Shang
  • Hongying Liu
  • Liang Wan
  • Wei Feng
  • Yanming Hui

While model quantization has become pivotal for deploying super-resolution (SR) networks on mobile devices, existing works focus on quantization methods only for image super-resolution. Different from image SR quantization, the temporal error propagation, shared temporal parameterization, and temporal metric mismatch significantly degrade the quantization performance of a video SR model. To address these issues, we propose the first quantization method, QBasicVSR, for video super-resolution. A novel temporal awareness adaptation post-training quantization (PTQ) framework for video super-resolution with the flow-gradient video bit adaptation and temporal shared layer bit adaptation is presented. Moreover, we put forward a novel fine-tuning method for VSR with the supervision of the full-precision model. Our method achieves extraordinary performance with state-of-the-art efficient VSR approaches, delivering up to $\times$200 faster processing speed while utilizing only 1/8 of the GPU resources. Additionally, extensive experiments demonstrate that the proposed method significantly outperforms existing PTQ algorithms on various datasets. For instance, it attains a 2. 53 dB increase on the UDM10 benchmark when quantizing BasicVSR to 4-bit with 100 unlabeled video clips. The code and models will be released on GitHub.

PDF Details

AAAI Conference 2025 Conference Paper

Scalable Federated One-Step Multi-View Clustering with Tensorized Regularization

  • Wei Feng
  • Danting Liu
  • Qianqian Wang
  • Wenqi Liang
  • Zheng Yan

Multi-view clustering (MVC) methods have garnered considerable attention within centralized data frameworks. However, real-world multi-view data are often collected and stored by different organizations, complicating the practical deployment of MVC and motivating the emergence of federated multi-view clustering (FMVC). Existing FMVC approaches typically necessitate post-processing to derive clustering labels and confront challenges in effectively exploring the complementary and consistent information across multi-view data residing in different entities. To address these limitations, we propose a novel framework termed Scalable Federated One-Step Multi-View Clustering with Tensorized Regularization (SFOMVC-TR). This framework facilitates one-step clustering at each client and employs tensor learning to capture consistent and complementary information through a centralized server. Additionally, it adopts anchor graphs to enhance clustering efficiency and scalability in high-dimensional data. By incorporating a Lp,q sparse regularization on the projection matrix, SFOMVC-TR enables the direct projection of anchors into clustering assignments to mitigate redundancy. A federated optimization framework is developed to support collaborative and privacy-preserving training under the coordination of the server. Extensive experiments on multiple datasets validate the privacy and effectiveness of our method.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Tensorial Multi-view Clustering with Deep Anchor Graph Projection

  • Wei Feng
  • Dongyuvan Wei
  • Qianqian Wang
  • Bo Dong

Multi-view clustering (MVC) has emerged as an important unsupervised multi-view learning method that leverages consistent and complementary information to enhance clustering performance. Recently, tensorized MVC, which processes multi-view data as a tensor to capture their cross-view information, has received considerable attention. However, existing tensorized MVC methods generally overlook deep structures within each view and rely on post-processing to derive clustering results, leading to potential information loss and degraded performance. To address these issues, we develop Tensorial Multi-view Clustering with Deep Anchor Graph Projection (TMVC-DAGP), which performs deep projection on the anchor graph, thus improving model scalability. Besides, we utilize a sparsity regularization to eliminate the redundancy and enforce the projected anchor graph to retain a clear clustering structure. Furthermore, TMVC-DAGP leverages weighted Tensor Schatten $p$-norm to exploit the consistent and complementary information. Extensive experiments on multiple datasets demonstrate TMVC-DAGP's effectiveness and superiority.

PDF Details DOI

TCS Journal 2025 Journal Article

The diagnosability of multiprocessor systems under the enhanced comparison model

  • Shiying Wang
  • Wei Feng

Diagnosability of a multiprocessor system G is a fundamental topic. The g -extra diagnosability of G requires that every fault-free component in G − F has at least g + 1 vertices, while the g -good-neighbor diagnosability requires that every fault-free vertex has at least g fault-free neighbors. In this paper, we first propose a new model, the enhanced comparison (EC) model on G, and establish the relationship between the g -extra diagnosability and the g -good-neighbor diagnosability of G. Second, we present and prove necessary and sufficient conditions for the g -extra diagnosability of G, and determine the g -extra diagnosability of the hyper Petersen network. Finally, we present and prove necessary and sufficient conditions for the g -good-neighbor diagnosability of G, and determine the g -good-neighbor diagnosability of the star graph.

Details DOI

ICLR Conference 2025 Conference Paper

TorchTitan: One-stop PyTorch native solution for production ready LLM pretraining

  • Wanchao Liang
  • Tianyu Liu
  • Less Wright
  • Will Constable
  • Andrew Gu
  • Chien-Chin Huang
  • Iris Zhang
  • Wei Feng

The development of large language models (LLMs) has been instrumental in advancing state-of-the-art natural language processing applications. Training LLMs with billions of parameters and trillions of tokens requires sophisticated distributed systems that enable composing and comparing several state-of-the-art techniques in order to efficiently scale across thousands of accelerators. However, existing solutions are complex, scattered across multiple libraries/repositories, lack interoperability, and are cumbersome to maintain. Thus, curating and empirically comparing training recipes requires non-trivial engineering effort. This paper introduces **TORCHTITAN**$^1$, a PyTorch-native distributed training system that unifies and advances state-of-the-art techniques, streamlining integration and reducing engineering overhead. TORCHTITAN enables seamless application of 4D parallelism in a modular and composable manner, while featuring elastic scaling to adapt to changing computational requirements. The system provides comprehensive logging, efficient checkpointing, and debugging tools, ensuring production-ready training. Moreover, TORCHTITAN incorporates innovative hardware-software co-designed solutions, leveraging cutting-edge features like Float8 training and SymmetricMemory to maximize hardware utilization. As a flexible experimental test bed, TORCHTITAN facilitates the curation and comparison of custom recipes for diverse training contexts. By leveraging TORCHTITAN, we developed optimized training recipes for the Llama 3.1 family and provide actionable guidance on selecting and combining distributed training techniques to maximize training efficiency, based on our hands-on experiences. We thoroughly assess TORCHTITAN on the Llama 3.1 family of LLMs, spanning 8 billion to 405 billion parameters, and showcase its exceptional performance, modular composability, and elastic scalability. By stacking training optimizations, we demonstrate accelerations ranging from 65.08% on Llama 3.1 8B at 128 GPU scale (1D), 12.59% on Llama 3.1 70B at 256 GPU scale (2D), to 30% on Llama 3.1 405B at 512 GPU scale (3D) on NVIDIA H100 GPUs over optimized baselines. We also demonstrate the effectiveness of 4D parallelism in enabling long context training. $^1$ GitHub: [https://github.com/pytorch/torchtitan](https://github.com/pytorch/torchtitan)

Details

AAAI Conference 2024 Conference Paper

COMMA: Co-articulated Multi-Modal Learning

  • Lianyu Hu
  • Liqing Gao
  • Zekang Liu
  • Chi-Man Pun
  • Wei Feng

Pretrained large-scale vision-language models such as CLIP have demonstrated excellent generalizability over a series of downstream tasks. However, they are sensitive to the variation of input text prompts and need a selection of prompt templates to achieve satisfactory performance. Recently, various methods have been proposed to dynamically learn the prompts as the textual inputs to avoid the requirements of laboring hand-crafted prompt engineering in the fine-tuning process. We notice that these methods are suboptimal in two aspects. First, the prompts of the vision and language branches in these methods are usually separated or uni-directionally correlated. Thus, the prompts of both branches are not fully correlated and may not provide enough guidance to align the representations of both branches. Second, it's observed that most previous methods usually achieve better performance on seen classes but cause performance degeneration on unseen classes compared to CLIP. This is because the essential generic knowledge learned in the pretraining stage is partly forgotten in the fine-tuning process. In this paper, we propose Co-Articulated Multi-Modal Learning (COMMA) to handle the above limitations. Especially, our method considers prompts from both branches to generate the prompts to enhance the representation alignment of both branches. Besides, to alleviate forgetting about the essential knowledge, we minimize the feature discrepancy between the learned prompts and the embeddings of hand-crafted prompts in the pre-trained CLIP in the late transformer layers. We evaluate our method across three representative tasks of generalization to novel classes, new target datasets and unseen domain shifts. Experimental results demonstrate the superiority of our method by exhibiting a favorable performance boost upon all tasks with high efficiency. Code is available at https://github.com/hulianyuyy/COMMA.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Deep Correlated Prompting for Visual Recognition with Missing Modalities

  • Lianyu Hu
  • Tongkai Shi
  • Wei Feng
  • Fanhua Shang
  • Liang Wan

Large-scale multimodal models have shown excellent performance over a series of tasks powered by the large corpus of paired multimodal training data. Generally, they are always assumed to receive modality-complete inputs. However, this simple assumption may not always hold in the real world due to privacy constraints or collection difficulty, where models pretrained on modality-complete data easily demonstrate degraded performance on missing-modality cases. To handle this issue, we refer to prompt learning to adapt large pretrained multimodal models to handle missing-modality scenarios by regarding different missing cases as different types of input. Instead of only prepending independent prompts to the intermediate layers, we present to leverage the correlations between prompts and input features and excavate the relationships between different layers of prompts to carefully design the instructions. We also incorporate the complementary semantics of different modalities to guide the prompting design for each modality. Extensive experiments on three commonly-used datasets consistently demonstrate the superiority of our method compared to the previous approaches upon different missing scenarios. Plentiful ablations are further given to show the generalizability and reliability of our method upon different modality-missing ratios and types.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Efficient Federated Multi-View Clustering with Integrated Matrix Factorization and K-Means

  • Wei Feng
  • Zhenwei Wu
  • Qianqian Wang
  • Bo Dong
  • Zhiqiang Tao
  • Quanxue Gao

Multi-view clustering is a popular unsupervised multi-view learning method. Real-world multi-view data are often distributed across multiple entities, presenting a challenge for performing multi-view clustering. Federated learning provides a solution by enabling multiple entities to collaboratively train a global model. However, existing federated multi-view clustering methods usually conduct feature extraction and clustering in separate steps, potentially leading to a degradation in clustering performance. To address this issue and for the sake of efficiency, we propose a novel Federated Multi-View Clustering method with Integrated Matrix Factorization and K-Means (FMVC-IMK), which integrates matrix factorization and multi-view K-means into one step. Additionally, an adaptive weight is employed to balance the influence of data from each view. FMVC-IMK further incorporates a graph-based regularizer to preserve the original data's geometric structure within the learned global clustering structure. We also develop a federated optimization approach to collaboratively learn a global clustering result without sharing any original data. Experimental results on multiple datasets demonstrate the effectiveness of FMVC-IMK.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Federated Multi-View Clustering via Tensor Factorization

  • Wei Feng
  • Zhenwei Wu
  • Qianqian Wang
  • Bo Dong
  • Zhiqiang Tao
  • Quanxue Gao

Multi-view clustering is an effective method to process massive unlabeled multi-view data. Since data of different views may be collected and held by different parties, it becomes impractical to train a multi-view clustering model in a centralized way, for the sake of privacy. However, federated multi-view clustering is challenging because multi-view learning has to consider the complementary and consistent information between each view distributed across different clients. For another, efficiency is highly expected in federated scenarios. Therefore, we propose a novel federated multi-view clustering method with tensor factorization (TensorFMVC), which is built based on K-means and hence is more efficient. Besides, TensorFMVC avoids initializing centroids to address the performance degradation of K-means due to its sensitivity to centroid initialization. A three-order tensor stacked by cluster assignment matrices is introduced to exploit the complementary information and spatial structure of different views. Furthermore, we divide the optimization into several subproblems and develop a federated optimization approach to support cooperative model training. Extensive experiments on several datasets demonstrate that our proposed method exhibits superior performance in federated multi-view clustering.

PDF Details DOI

EAAI Journal 2024 Journal Article

Hyperspectral band selection via region-wise latent feature fusion and graph filter embedded subspace clustering

  • Wei Feng
  • Minhui Wang
  • Chang Tang
  • Weiying Xie
  • Xianju Li
  • Xiao Zheng
  • Jiangfeng Xu

Hyperspectral band selection plays a crucial role in reducing dimensionality, extracting relevant features, and improving computational efficiency in hyperspectral data analysis. Although numerous band selection methods have emerged in recent years, there remains a significant gap in exploring spatial structures and the diversity of ground objects. In this paper, we propose a region-wise latent feature fusion and graph filter embedded subspace clustering approach to address the band selection problem. Specifically, we segment the original hyperspectral image into diverse homogeneous regions using entropy rate superpixel segmentation. Next, we fuse the features from these regions into a consensus low-dimensional latent space, effectively capturing spatial information. To explore the spectral correlation among all bands, we employ a self-representation subspace clustering model on the fused latent features. Additionally, we apply a graph filter to the raw region-wise features to reduce redundant and noisy information present in the original data. By integrating these concerns into a unified framework, we facilitate mutual reinforcement among the learning sub-tasks. We conduct experiments on four public hyperspectral datasets, demonstrating a 1. 93%, 0. 50%, 0. 76%, and 0. 94% accuracy enhancement over current state-of-the-art methods. Additionally, our proposed optimization method enables the model to converge rapidly in under 15 iterations.

Details DOI

AAAI Conference 2024 Conference Paper

Long-Tailed Learning as Multi-Objective Optimization

  • Weiqi Li
  • Fan Lyu
  • Fanhua Shang
  • Liang Wan
  • Wei Feng

Real-world data is extremely imbalanced and presents a long-tailed distribution, resulting in models biased towards classes with sufficient samples and performing poorly on rare classes. Recent methods propose to rebalance classes but they undertake the seesaw dilemma (what is increasing performance on tail classes may decrease that of head classes, and vice versa). In this paper, we argue that the seesaw dilemma is derived from the gradient imbalance of different classes, in which gradients of inappropriate classes are set to important for updating, thus prone to overcompensation or undercompensation on tail classes. To achieve ideal compensation, we formulate long-tailed recognition as a multi-objective optimization problem, which fairly respects the contributions of head and tail classes simultaneously. For efficiency, we propose a Gradient-Balancing Grouping (GBG) strategy to gather the classes with similar gradient directions, thus approximately making every update under a Pareto descent direction. Our GBG method drives classes with similar gradient directions to form a more representative gradient and provides ideal compensation to the tail classes. Moreover, we conduct extensive experiments on commonly used benchmarks in long-tailed learning and demonstrate the superiority of our method over existing SOTA methods. Our code is released at https://github.com/WickyLee1998/GBG_v1.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Partial Multi-View Clustering via Self-Supervised Network

  • Wei Feng
  • Guoshuai Sheng
  • Qianqian Wang
  • Quanxue Gao
  • Zhiqiang Tao
  • Bo Dong

Partial multi-view clustering is a challenging and practical research problem for data analysis in real-world applications, due to the potential data missing issue in different views. However, most existing methods have not fully explored the correlation information among various incomplete views. In addition, these existing clustering methods always ignore discovering discriminative features inside the data itself in this unsupervised task. To tackle these challenges, we propose Partial Multi-View Clustering via Self-Supervised \textbf{N}etwork (PVC-SSN) in this paper. Specifically, we employ contrastive learning to obtain a more discriminative and consistent subspace representation, which is guided by a self-supervised module. Self-supervised learning can exploit effective cluster information through the data itself to guide the learning process of clustering tasks. Thus, it can pull together embedding features from the same cluster and push apart these from different clusters. Extensive experiments on several benchmark datasets show that the proposed PVC-SCN method outperforms several state-of-the-art clustering methods.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Reconstruction Weighting Principal Component Analysis with Fusion Contrastive Learning

  • Qianqian Wang
  • Meiling Liu
  • Wei Feng
  • Mengping Jiang
  • Haiming Xu
  • Quanxue Gao

Principal component analysis (PCA) is a popular unsupervised dimensionality reduction method to extract the principal components of data. However, there are two problems with the existing PCA: (1) Traditional PCA methods treat each sample equally and ignore sample differences. (2) They fail to extract the discriminative features required by recognition tasks. To overcome these problems, we incorporate contrastive learning to develop a novel weighted PCA algorithm. Specifically, our method weights the reconstruction error of individual samples to reduce the influence of outliers. Besides, it integrates contrastive learning into PCA to increase inter-class distances and reduce intra-class distance, which helps to improve PCA's discriminative capability. We further develop an unsupervised strategy to select positive and negative samples, which eliminates pseudo-negative samples guided by clustering labels. Specifically, it employs confidence level to distinguish positive and negative samples. Consequently, our method achieves higher recognition accuracy on benchmark datasets.

PDF Details DOI

AAAI Conference 2024 Conference Paper

SAVSR: Arbitrary-Scale Video Super-Resolution via a Learned Scale-Adaptive Network

  • Zekun Li
  • Hongying Liu
  • Fanhua Shang
  • Yuanyuan Liu
  • Liang Wan
  • Wei Feng

Deep learning-based video super-resolution (VSR) networks have gained significant performance improvements in recent years. However, existing VSR networks can only support a fixed integer scale super-resolution task, and when we want to perform VSR at multiple scales, we need to train several models. This implementation certainly increases the consumption of computational and storage resources, which limits the application scenarios of VSR techniques. In this paper, we propose a novel Scale-adaptive Arbitrary-scale Video Super-Resolution network (SAVSR), which is the first work focusing on spatial VSR at arbitrary scales including both non-integer and asymmetric scales. We also present an omni-dimensional scale-attention convolution, which dynamically adapts according to the scale of the input to extract inter-frame features with stronger representational power. Moreover, the proposed spatio-temporal adaptive arbitrary-scale upsampling performs VSR tasks using both temporal features and scale information. And we design an iterative bi-directional architecture for implicit feature alignment. Experiments at various scales on the benchmark datasets show that the proposed SAVSR outperforms state-of-the-art (SOTA) methods at non-integer and asymmetric scales. The source code is available at https://github.com/Weepingchestnut/SAVSR.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding

  • Houlun Chen
  • Xin Wang
  • Hong Chen
  • Zeyang Zhang
  • Wei Feng
  • Bin Huang
  • Jia Jia
  • Wenwu Zhu

Existing Video Corpus Moment Retrieval (VCMR) is limited to coarse-grained understanding that hinders precise video moment localization when given fine-grained queries. In this paper, we propose a more challenging fine-grained VCMR benchmark requiring methods to localize the best-matched moment from the corpus with other partially matched candidates. To improve the dataset construction efficiency and guarantee high-quality data annotations, we propose VERIFIED, an automatic \underline{V}id\underline{E}o-text annotation pipeline to generate captions with \underline{R}el\underline{I}able \underline{FI}n\underline{E}-grained statics and \underline{D}ynamics. Specifically, we resort to large language models (LLM) and large multimodal models (LMM) with our proposed Statics and Dynamics Enhanced Captioning modules to generate diverse fine-grained captions for each video. To filter out the inaccurate annotations caused by the LLM hallucination, we propose a Fine-Granularity Aware Noise Evaluator where we fine-tune a video foundation model with disturbed hard-negatives augmented contrastive and matching losses. With VERIFIED, we construct a more challenging fine-grained VCMR benchmark containing Charades-FIG, DiDeMo-FIG, and ActivityNet-FIG which demonstrate a high level of annotation quality. We evaluate several state-of-the-art VCMR models on the proposed dataset, revealing that there is still significant scope for fine-grained video understanding in VCMR.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understanding

  • Ming Hu
  • Lin Wang
  • Siyuan Yan
  • Don Ma
  • Qingli Ren
  • Peng Xia
  • Wei Feng
  • Peibo Duan

The application of deep learning to nursing procedure activity understanding has the potential to greatly enhance the quality and safety of nurse-patient interactions. By utilizing the technique, we can facilitate training and education, improve quality control, and enable operational compliance monitoring. However, the development of automatic recognition systems in this field is currently hindered by the scarcity of appropriately labeled datasets. The existing video datasets pose several limitations: 1) these datasets are small-scale in size to support comprehensive investigations of nursing activity; 2) they primarily focus on single procedures, lacking expert-level annotations for various nursing procedures and action steps; and 3) they lack temporally localized annotations, which prevents the effective localization of targeted actions within longer video sequences. To mitigate these limitations, we propose NurViD, a large video dataset with expert-level annotation for nursing procedure activity understanding. NurViD consists of over 1. 5k videos totaling 144 hours, making it approximately four times longer than the existing largest nursing activity datasets. Notably, it encompasses 51 distinct nursing procedures and 177 action steps, providing a much more comprehensive coverage compared to existing datasets that primarily focus on limited procedures. To evaluate the efficacy of current deep learning methods on nursing activity understanding, we establish three benchmarks on NurViD: procedure recognition on untrimmed videos, procedure and action recognition on trimmed videos, and action detection. Our benchmark and code will be available at https: //github. com/minghu0830/NurViD-benchmark.

PDF Details

NeurIPS Conference 2023 Conference Paper

OFCOURSE: A Multi-Agent Reinforcement Learning Environment for Order Fulfillment

  • Yiheng Zhu
  • Yang Zhan
  • Xuankun Huang
  • Yuwei Chen
  • Yujie Chen
  • Jiangwen Wei
  • Wei Feng
  • Yinzhi Zhou

The dramatic growth of global e-commerce has led to a surge in demand for efficient and cost-effective order fulfillment which can increase customers' service levels and sellers' competitiveness. However, managing order fulfillment is challenging due to a series of interdependent online sequential decision-making problems. To clear this hurdle, rather than solving the problems separately as attempted in some recent researches, this paper proposes a method based on multi-agent reinforcement learning to integratively solve the series of interconnected problems, encompassing order handling, packing and pickup, storage, order consolidation, and last-mile delivery. In particular, we model the integrated problem as a Markov game, wherein a team of agents learns a joint policy via interacting with a simulated environment. Since no simulated environment supporting the complete order fulfillment problem exists, we devise Order Fulfillment COoperative mUlti-agent Reinforcement learning Scalable Environment (OFCOURSE) in the OpenAI Gym style, which allows reproduction and re-utilization to build customized applications. By constructing the fulfillment system in OFCOURSE, we optimize a joint policy that solves the integrated problem, facilitating sequential order-wise operations across all fulfillment units and minimizing the total cost of fulfilling all orders within the promised time. With OFCOURSE, we also demonstrate that the joint policy learned by multi-agent reinforcement learning outperforms the combination of locally optimal policies. The source code of OFCOURSE is available at: https: //github. com/GitYiheng/ofcourse.

PDF Details

NeurIPS Conference 2023 Conference Paper

Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation

  • Tingliang Feng
  • Hao Shi
  • Xueyang Liu
  • Wei Feng
  • Liang Wan
  • Yanlin Zhou
  • Di Lin

Many methods of semantic image segmentation have borrowed the success of open compound domain adaptation. They minimize the style gap between the images of source and target domains, more easily predicting the accurate pseudo annotations for target domain's images that train segmentation network. The existing methods globally adapt the scene style of the images, whereas the object styles of different categories or instances are adapted improperly. This paper proposes the Object Style Compensation, where we construct the Object-Level Discrepancy Memory with multiple sets of discrepancy features. The discrepancy features in a set capture the style changes of the same category's object instances adapted from target to source domains. We learn the discrepancy features from the images of source and target domains, storing the discrepancy features in memory. With this memory, we select appropriate discrepancy features for compensating the style information of the object instances of various categories, adapting the object styles to a unified style of source domain. Our method enables a more accurate computation of the pseudo annotations for target domain's images, thus yielding state-of-the-art results on different datasets.

PDF Details

AAAI Conference 2023 Conference Paper

Self-Emphasizing Network for Continuous Sign Language Recognition

  • Lianyu Hu
  • Liqing Gao
  • Zekang Liu
  • Wei Feng

Hand and face play an important role in expressing sign language. Their features are usually especially leveraged to improve system performance. However, to effectively extract visual representations and capture trajectories for hands and face, previous methods always come at high computations with increased training complexity. They usually employ extra heavy pose-estimation networks to locate human body keypoints or rely on additional pre-extracted heatmaps for supervision. To relieve this problem, we propose a self-emphasizing network (SEN) to emphasize informative spatial regions in a self-motivated way, with few extra computations and without additional expensive supervision. Specifically, SEN first employs a lightweight subnetwork to incorporate local spatial-temporal features to identify informative regions, and then dynamically augment original features via attention maps. It's also observed that not all frames contribute equally to recognition. We present a temporal self-emphasizing module to adaptively emphasize those discriminative frames and suppress redundant ones. A comprehensive comparison with previous methods equipped with hand and face features demonstrates the superiority of our method, even though they always require huge computations and rely on expensive extra supervision. Remarkably, with few extra computations, SEN achieves new state-of-the-art accuracy on four large-scale datasets, PHOENIX14, PHOENIX14-T, CSL-Daily, and CSL. Visualizations verify the effects of SEN on emphasizing informative spatial and temporal features. Code is available at https://github.com/hulianyuyy/SEN_CSLR

PDF Details DOI

AAAI Conference 2023 Conference Paper

Social Relation Reasoning Based on Triangular Constraints

  • Yunfei Guo
  • Fei Yin
  • Wei Feng
  • Xudong Yan
  • Tao Xue
  • Shuqi Mei
  • Cheng-Lin Liu

Social networks are essentially in a graph structure where persons act as nodes and the edges connecting nodes denote social relations. The prediction of social relations, therefore, relies on the context in graphs to model the higher-order constraints among relations, which has not been exploited sufficiently by previous works, however. In this paper, we formulate the paradigm of the higher-order constraints in social relations into triangular relational closed-loop structures, i.e., triangular constraints, and further introduce the triangular reasoning graph attention network (TRGAT). Our TRGAT employs the attention mechanism to aggregate features with triangular constraints in the graph, thereby exploiting the higher-order context to reason social relations iteratively. Besides, to acquire better feature representations of persons, we introduce node contrastive learning into relation reasoning. Experimental results show that our method outperforms existing approaches significantly, with higher accuracy and better consistency in generating social relation graphs.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment

  • Wei Feng
  • Lie Ju
  • Lin Wang
  • Kaimin Song
  • Xin Zhao
  • Zongyuan Ge

Generalizing a deep learning model to new domains is crucial for computer-aided medical diagnosis systems. Most existing unsupervised domain adaptation methods have made significant progress in reducing the domain distribution gap through adversarial training. However, these methods may still produce overconfident but erroneous results on unseen target images. This paper proposes a new unsupervised domain adaptation framework for cross-modality medical image segmentation. Specifically, We first introduce two data augmentation approaches to generate two sets of semantics-preserving augmented images. Based on the model's predictive consistency on these two sets of augmented images, we identify reliable and unreliable pixels. We then perform a selective entropy constraint: we minimize the entropy of reliable pixels to increase their confidence while maximizing the entropy of unreliable pixels to reduce their confidence. Based on the identified reliable and unreliable pixels, we further propose an adaptive semantic alignment module which performs class-level distribution adaptation by minimizing the distance between same class prototypes between domains, where unreliable pixels are removed to derive more accurate prototypes. We have conducted extensive experiments on the cross-modality cardiac structure segmentation task. The experimental results show that the proposed method significantly outperforms the state-of-the-art comparison algorithms. Our code and data are available at https://github.com/fengweie/SE_ASA.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Cross-Image Context for Single Image Inpainting

  • Tingliang Feng
  • Wei Feng
  • Weiqi Li
  • Di Lin

Visual context is of crucial importance for image inpainting. The contextual information captures the appearance and semantic correlation between the image regions, helping to propagate the information of the complete regions for reasoning the content of the corrupted regions. Many inpainting methods compute the visual context based on the regions within the single image. In this paper, we propose the Cross-Image Context Memory (CICM) for learning and using the cross-image context to recover the corrupted regions. CICM consists of multiple sets of the cross-image representations learned from the image regions with different visual patterns. The regional representations are learned across different images, thus providing richer context that benefit the inpainting task. The experimental results demonstrate the effectiveness and generalization of CICM, which achieves state-of-the-art performances on various datasets for single image inpainting.

PDF Details

NeurIPS Conference 2022 Conference Paper

Exploring Example Influence in Continual Learning

  • Qing Sun
  • Fan Lyu
  • Fanhua Shang
  • Wei Feng
  • Liang Wan

Continual Learning (CL) sequentially learns new tasks like human beings, with the goal to achieve better Stability (S, remembering past tasks) and Plasticity (P, adapting to new tasks). Due to the fact that past training data is not available, it is valuable to explore the influence difference on S and P among training examples, which may improve the learning pattern towards better SP. Inspired by Influence Function (IF), we first study example influence via adding perturbation to example weight and computing the influence derivation. To avoid the storage and calculation burden of Hessian inverse in neural networks, we propose a simple yet effective MetaSP algorithm to simulate the two key steps in the computation of IF and obtain the S- and P-aware example influence. Moreover, we propose to fuse two kinds of example influence by solving a dual-objective optimization problem, and obtain a fused influence towards SP Pareto optimality. The fused influence can be used to control the update of model and optimize the storage of rehearsal. Empirical results show that our algorithm significantly outperforms state-of-the-art methods on both task- and class-incremental benchmark CL datasets.

PDF Details

AAAI Conference 2022 Conference Paper

Regularized Modal Regression on Markov-Dependent Observations: A Theoretical Assessment

  • Tieliang Gong
  • Yuxin Dong
  • Hong Chen
  • Wei Feng
  • Bo Dong
  • Chen Li

Modal regression, a widely used regression protocol, has been extensively investigated in statistical and machine learning communities due to its robustness to outliers and heavy-tailed noises. Understanding modal regression’s theoretical behavior can be fundamental in learning theory. Despite significant progress in characterizing its statistical property, the majority of the results are based on the assumption that samples are independent and identical distributed (i. i. d.), which is too restrictive for real-world applications. This paper concerns the statistical property of regularized modal regression (RMR) within an important dependence structure - Markov dependent. Specifically, we establish the upper bound for RMR estimator under moderate conditions and give an explicit learning rate. Our results show that the Markov dependence impacts on the generalization error in the way that sample size would be discounted by a multiplicative factor depending on the spectral gap of underlying Markov chain. This result shed a new light on characterizing the theoretical underpinning for robust regression.

PDF Details

TCS Journal 2022 Journal Article

The diagnosability of wheel networks with the condition: 2-extra

  • Wei Feng
  • Shiying Wang

Diagnosability is an essential parameter for a multiprocessor system when measuring the reliability of an interconnection network. The g-extra diagnosability of a system G denoted by t ˜ g ( G ) was introduced by Zhang et al. , which constrains that each fault-free component G i owns | V ( G i ) | ≥ g + 1 fault-free nodes. The n-dimensional wheel network C W n has advantages as a typical topological structure in interconnection networks. The paper establishes the 2-extra diagnosability of C W n to be 6 n − 10 for n ≥ 6 when the PMC model and MM⁎ model are employed.

Details DOI

AAAI Conference 2021 Conference Paper

EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining

  • Qing Guo
  • Jingyang Sun
  • Felix Juefei-Xu
  • Lei Ma
  • Xiaofei Xie
  • Wei Feng
  • Yang Liu
  • Jianjun Zhao

Single-image deraining is rather challenging due to the unknown rain model. Existing methods often make specific assumptions of the rain model, which can hardly cover many diverse circumstances in the real world, compelling them to employ complex optimization or progressive refinement. This, however, significantly affects these methods’ efficiency and effectiveness for many efficiency-critical applications. To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i. e. , Efficient- DeRain, which is able to process a rainy image within 10 ms (i. e. , around 6 ms on average), over 80 times faster than the state-of-the-art method (i. e. , RCDNet), while achieving similar de-rain effects. We first propose the novel pixel-wise dilation filtering. In particular, a rainy image is filtered with the pixel-wise kernels estimated from a kernel prediction network, by which suitable multi-scale kernels for each pixel can be efficiently predicted. Then, to eliminate the gap between synthetic and real data, we further propose an effective data augmentation method (i. e. , RainMix) that helps to train network for handling real rainy images. We perform comprehensive evaluation on both synthetic and realworld rainy datasets to demonstrate the effectiveness and efficiency of our method. We release the model and code in https: //github. com/tsingqguo/efficientderain. git.

PDF Details

AAAI Conference 2021 Conference Paper

Multi-Domain Multi-Task Rehearsal for Lifelong Learning

  • Fan Lyu
  • Shuai Wang
  • Wei Feng
  • Zihan Ye
  • Fuyuan Hu
  • Song Wang

Rehearsal, seeking to remind the model by storing old knowledge in lifelong learning, is one of the most effective ways to mitigate catastrophic forgetting, i. e. , biased forgetting of previous knowledge when moving to new tasks. However, the old tasks of the most previous rehearsal-based methods suffer from the unpredictable domain shift when training the new task. This is because these methods always ignore two significant factors. First, the Data Imbalance between the new task and old tasks that makes the domain of old tasks prone to shift. Second, the Task Isolation among all tasks will make the domain shift toward unpredictable directions; To address the unpredictable domain shift, in this paper, we propose Multi- Domain Multi-Task (MDMT) rehearsal to train the old tasks and new task parallelly and equally to break the isolation among tasks. Specifically, a two-level angular margin loss is proposed to encourage the intra-class/task compactness and inter-class/task discrepancy, which keeps the model from domain chaos. In addition, to further address domain shift of the old tasks, we propose an optional episodic distillation loss on the memory to anchor the knowledge for each old task. Experiments on benchmark datasets validate the proposed approach can effectively mitigate the unpredictable domain shift.

PDF Details

TCS Journal 2021 Journal Article

Structure connectivity and substructure connectivity of wheel networks

  • Wei Feng
  • Shiying Wang

Consider a graph G and its connected subgraph T. The T-structure connectivity κ ( G; T ) of G is the cardinality of a minimum set of subgraphs in G, whose removal disconnects G and each element in the set is isomorphic to T. The T-substructure connectivity κ s ( G; T ) of G is the cardinality of a minimum set of subgraphs in G, whose removal disconnects G and each element in the set is isomorphic to a connected subgraph of T. In G, the standard connectivity κ ( G ) is regarded as a simplification of both κ ( G; T ) and κ s ( G; T ). The wheel network, denoted by C W n, is an attractive interconnected network prototype for multiple CPU systems. In this paper, we determine κ ( C W n; P 2 k + 1 ) (resp. κ s ( C W n; P 2 k + 1 ) ) for n ≥ 5 and k + 1 ≤ 2 n − 4, κ ( C W n; P 2 k ) (resp. κ s ( C W n; P 2 k ) ) for n ≥ 6 and k ≤ 2 n − 4 and a lower bound of κ ( C W n; C 2 k ) (resp. κ s ( C W n; C 2 k ) ) for n ≥ 6 and k ≤ 2 n − 4.

Details DOI

AAAI Conference 2020 Conference Paper

Complementary-View Multiple Human Tracking

  • Ruize Han
  • Wei Feng
  • Jiewen Zhao
  • Zicheng Niu
  • Yujun Zhang
  • Liang Wan
  • Song Wang

The global trajectories of targets on ground can be well captured from a top view in a high altitude, e. g. , by a dronemounted camera, while their local detailed appearances can be better recorded from horizontal views, e. g. , by a helmet camera worn by a person. This paper studies a new problem of multiple human tracking from a pair of top- and horizontalview videos taken at the same time. Our goal is to track the humans in both views and identify the same person across the two complementary views frame by frame, which is very challenging due to very large field of view difference. In this paper, we model the data similarity in each view using appearance and motion reasoning and across views using appearance and spatial reasoning. Combing them, we formulate the proposed multiple human tracking as a joint optimization problem, which can be solved by constrained integer programming. We collect a new dataset consisting of top- and horizontal-view video pairs for performance evaluation and the experimental results show the effectiveness of the proposed method.

PDF Details

ICLR Conference 2020 Conference Paper

Dynamically Pruned Message Passing Networks for Large-scale Knowledge Graph Reasoning

  • Xiaoran Xu
  • Wei Feng
  • Yunsheng Jiang
  • Xiaohui Xie
  • Zhiqing Sun
  • Zhi-Hong Deng 0001

We propose Dynamically Pruned Message Passing Networks (DPMPN) for large-scale knowledge graph reasoning. In contrast to existing models, embedding-based or path-based, we learn an input-dependent subgraph to explicitly model a sequential reasoning process. Each subgraph is dynamically constructed, expanding itself selectively under a flow-style attention mechanism. In this way, we can not only construct graphical explanations to interpret prediction, but also prune message passing in Graph Neural Networks (GNNs) to scale with the size of graphs. We take the inspiration from the consciousness prior proposed by Bengio to design a two-GNN framework to encode global input-invariant graph-structured representation and learn local input-dependent one coordinated by an attention module. Experiments show the reasoning capability in our model that is providing a clear graphical explanation as well as predicting results accurately, outperforming most state-of-the-art methods in knowledge base completion tasks.

Details

NeurIPS Conference 2020 Conference Paper

Watch out! Motion is Blurring the Vision of Your Deep Neural Networks

  • Qing Guo
  • Felix Juefei-Xu
  • Xiaofei Xie
  • Lei Ma
  • Jian Wang
  • Bing Yu
  • Wei Feng
  • Yang Liu

The state-of-the-art deep neural networks (DNNs) are vulnerable against adversarial examples with additive random-like noise perturbations. While such examples are hardly found in the physical world, the image blurring effect caused by object motion, on the other hand, commonly occurs in practice, making the study of which greatly important especially for the widely adopted real-time image processing tasks (e. g. , object detection, tracking). In this paper, we initiate the first step to comprehensively investigate the potential hazards of blur effect for DNN, caused by object motion. We propose a novel adversarial attack method that can generate visually natural motion-blurred adversarial examples, named motion-based adversarial blur attack (ABBA). To this end, we first formulate the kernel-prediction-based attack where an input image is convolved with kernels in a pixel-wise way, and the misclassification capability is achieved by tuning the kernel weights. To generate visually more natural and plausible examples, we further propose the saliency-regularized adversarial kernel prediction, where the salient region serves as a moving object, and the predicted kernel is regularized to achieve naturally visual effects. Besides, the attack is further enhanced by adaptively tuning the translations of object and background. A comprehensive evaluation on the NeurIPS'17 adversarial competition dataset demonstrates the effectiveness of ABBA by considering various kernel sizes, translations, and regions. The in-depth study further confirms that our method shows a more effective penetrating capability to the state-of-the-art GAN-based deblurring mechanisms compared with other blurring methods. We release the code to \url{https: //github. com/tsingqguo/ABBA}.

PDF Details

AAAI Conference 2019 Conference Paper

Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation

  • Wei Feng
  • Wentao Liu
  • Tong Li
  • Jing Peng
  • Chen Qian
  • Xiaolin Hu

Human-object interactions (HOI) recognition and pose estimation are two closely related tasks. Human pose is an essential cue for recognizing actions and localizing the interacted objects. Meanwhile, human action and their interacted objects’ localizations provide guidance for pose estimation. In this paper, we propose a turbo learning framework to perform HOI recognition and pose estimation simultaneously. First, two modules are designed to enforce message passing between the tasks, i. e. pose aware HOI recognition module and HOI guided pose estimation module. Then, these two modules form a closed loop to utilize the complementary information iteratively, which can be trained in an end-to-end manner. The proposed method achieves the state-of-the-art performance on two public benchmarks including Verbs in COCO (V-COCO) and HICO-DET datasets.

PDF Details

IJCAI Conference 2018 Conference Paper

Active Recurrence of Lighting Condition for Fine-Grained Change Detection

  • Qian Zhang
  • Wei Feng
  • Liang Wan
  • Fei-Peng Tian
  • Ping Tan

This paper addresses active lighting recurrence (ALR), a new problem that actively relocalizes a light source to physically reproduce the lighting condition for a same scene from single reference image. ALR is of great importance for fine-grained visual monitoring and change detection, because some phenomena or minute changes can only be clearly observed under particular lighting conditions. Hence, effective ALR should be able to online navigate a light source toward the target pose, which is challenging due to the complexity and diversity of real-world lighting \& imaging processes. We propose to use the simple parallel lighting as an analogy model and based on Lambertian law to compose an instant navigation ball for this purpose. We theoretically prove the feasibility of this ALR strategy for realistic near point light sources and its invariance to the ambiguity of normal \& lighting decomposition. Extensive quantitative experiments and challenging real-world tasks on fine-grained change monitoring of cultural heritages verify the effectiveness of our approach. We also validate its generality to non-Lambertian scenes.

PDF Details

AAAI Conference 2018 Conference Paper

Co-Saliency Detection Within a Single Image

  • Hongkai Yu
  • Kang Zheng
  • Jianwu Fang
  • Hao Guo
  • Wei Feng
  • Song Wang

Recently, saliency detection in a single image and co-saliency detection in multiple images have drawn extensive research interest in the vision community. In this paper, we investigate a new problem of co-saliency detection within a single image, i. e. , detecting within-image co-saliency. By identifying common saliency within an image, e. g. , highlighting multiple occurrences of an object class with similar appearance, this work can benefit many important applications, such as the detection of objects of interest, more robust object recognition, reduction of information redundancy, and animation synthesis. We propose a new bottom-up method to address this problem. Specifically, a large number of object proposals are first detected from the image. Then we develop an optimization algorithm to derive a set of proposal groups, each of which contains multiple proposals showing good common saliency in the original image. For each proposal group, we calculate a co-saliency map and then use a low-rank based algorithm to fuse the maps calculated from all the proposal groups for the final co-saliency map in the image. In the experiment, we collect a new dataset of 364 color images with within-image cosaliency. Experiment results show that the proposed method can better detect the within-image co-saliency than existing algorithms.

PDF Details

IJCAI Conference 2013 Conference Paper

Integrating Semantic Relatedness and Words' Intrinsic Features for Keyword Extraction

  • Wei Zhang
  • Wei Feng
  • Jianyong Wang

Keyword extraction attracts much attention for its significant role in various natural language processing tasks. While some existing methods for keyword extraction have considered using single type of semantic relatedness between words or inherent attributes of words, almost all of them ignore two important issues: 1) how to fuse multiple types of semantic relations between words into a uniform semantic measurement and automatically learn the weights of the edges between the words in the word graph of each document, and 2) how to integrate the relations between words and words’ intrinsic features into a unified model. In this work, we tackle the two issues based on the supervised random walk model. We propose a supervised ranking based method for keyword extraction, which is called SEAFARER1. It can not only automatically learn the weights of the edges in the unified graph of each document which includes multiple semantic relations but also combine the merits of semantic relations of edges and intrinsic attributes of nodes together. We conducted extensive experimental study on an established benchmark and the experimental results demonstrate that SEAFARER outperforms the state-of-the-art supervised and unsupervised methods.

PDF Details DOI

ICRA Conference 2012 Conference Paper

The robustness of scheduling policies in multi-product manufacturing systems with sequence-dependent setup times and finite buffers

  • Wei Feng
  • Li Zheng
  • Jingshan Li

In this paper, a continuous time Markov chain model is introduced to study multi-product manufacturing systems with sequence-dependent setup times and finite buffers under seven scheduling policies, i. e. , cyclic, shortest queue, shortest processing time, shortest overall time (including setup time and processing times), longest queue, longest processing time, and longest overall time. In manufacturing environments, optimal solution may not be applicable due to uncertainty and variation in system parameters. Therefore, in this paper, in addition to comparing the system throughput under different policies, we introduce the notion of robustness of scheduling policies. Specifically, a policy that can deliver good and stable performance resilient to variations in system parameters (such as buffer sizes, processing rates, setup times, etc.) is viewed as a “robust” policy. Numerical studies indicate that the cyclic and longest queue policies exhibit robustness in subject to parameter changes. This can provide production engineers a guideline in operation management.

Details
v2026.09.13