Author name cluster

Wei Luo

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

26 papers

2 author rows

AAAI Conference 2026 Conference Paper

Anomagic: Crossmodal Prompt-driven Zero-shot Anomaly Generation

Yuxin Jiang
Wei Luo
Hui Zhang
Qiyu Chen
Haiming Yao
Weiming Shen
Yunkang Cao

We propose Anomagic, a zero-shot anomaly generation method that produces semantically coherent anomalies without requiring any exemplar anomalies. By unifying both visual and textual cues through a crossmodal prompt encoding scheme, Anomagic leverages rich contextual information to steer an inpainting‐based generation pipeline. A subsequent contrastive refinement strategy enforces precise alignment between synthesized anomalies and their masks, thereby bolstering downstream anomaly detection accuracy. To facilitate training, we introduce AnomVerse, a collection of 12,987 anomaly–mask–caption triplets assembled from 13 publicly available datasets, where captions are automatically generated by multimodal large language models using structured visual prompts and template‐based textual hints. Extensive experiments demonstrate that Anomagic trained on AnomVerse can synthesize more realistic and varied anomalies than prior methods, yielding superior improvements in downstream anomaly detection. Furthermore, Anomagic can generate anomalies for any normal‐category image using user‐defined prompts, establishing a versatile foundation model for anomaly generation.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DeepWriter: A Multi-Agent Collaboration Framework for Information-rich Ultra-long Book Writing

Ming Wang
Minghao Hu
Xiuli Kang
Li He
Yu Tian
Chunming Liu
Han Shi
Zhunchen Luo

Long-form books are among the most information-rich and structurally complex forms of written content, often exceeding 100,000 words. While recent methods have enabled basic long-text generation, they remain limited in two key aspects: the inability to generate ultra-long content at book scale, and the lack of mechanisms for integrating rich factual information. To address these limitations, we propose DeepWriter, a multi-agent collaborative framework that follows a structured planning-then-generation paradigm. It first constructs a detailed book outline with narrative arcs and chapter semantics, then incrementally generates content conditioned on retrieved knowledge and contextual signals. DeepWriter supports controllable generation of full-length books exceeding 100,000 words, enriched with citations, trivia and images. To support evaluation beyond surface-level fluency, we introduce DeepWriter-Bench, a bilingual benchmark of 18 annotated books designed to assess book-scale coherence, richness, and factual grounding. Additionally, we propose BookScore, a unified 100-point metric for quantifying book maturity. Experimental results show that DeepWriter achieves a state-of-the-art BookScore of 80.92, consistently outperforming strong baselines.

PDF Details DOI

JBHI Journal 2026 Journal Article

Mining Global and Local Semantics From Unlabeled Spectra for Spectral Classification

Wei Luo
Haiming Yao
Ang Gao
Tao Zhou
Xue Wang

Non-destructive detection methods based on molecular vibrational spectroscopy are pivotal in fields such as analytical chemistry and medical diagnostics. Recent advances have integrated deep learning with vibrational spectroscopy, significantly enhancing spectral recognition accuracy. However, these methods often rely on large annotated spectral datasets, limiting their general applicability. To address this limitation, we propose a novel approach, G lobal and L ocal S emantics M ining (GLSM), which leverages self-supervised learning to capture the global and local semantic information of unlabeled spectra, obviating the need for extensive annotated data. We devise two proxy tasks: global semantic mining and local semantic mining. The global semantic mining task is based on the premise that different views of the same spectrum can be mutually transformed, enabling the model to capture domain-invariant features across various perspectives and thereby develop a global understanding of the spectral data. This, in turn, enhances the model’s robustness to variations in peak positions. Meanwhile, the local semantic mining task posits that noisy spectra can be reconstructed into noise-free spectra, thereby facilitating the extraction of local patterns and fine-grained details, such as subtle variations in peak intensities. By combining both self-supervised tasks, our model effectively captures the global and local semantic information of the spectrum. The pre-trained model can be fine-tuned with a limited amount of labeled homologous or heterologous spectral data for semi-supervised or transfer learning-based spectral classification. Extensive experiments on three datasets in semi-supervised and transfer learning-based spectral recognition tasks comprehensively validate the effectiveness of our GLSM method, demonstrating its significant potential for real-world spectral analysis applications.

Details DOI

AAAI Conference 2026 Conference Paper

Parameter-, Memory-, Time-Efficient Multi-Task Dense Vision Adaptation

Haiming Yao
Wei Luo
Qiyu Chen
Jianxing Liao
Wei You

While adapting pretrained vision models to downstream dense prediction tasks is widely used, current methods often overlook adaptation efficiency, especially in the context of multi-task learning (MTL). Although parameter-efficient fine-tuning (PEFT) methods can enhance parameter efficiency, broader aspects such as GPU memory and training time efficiency remain underexplored. In this paper, we propose a new paradigm that simultaneously achieves efficiency in Parameters, GPU Memory, and Training Time for Multi-Task Dense Vision Adaptation. Specifically, we propose a dual-branch framework, in which a frozen pretrained backbone serves as the generic main branch, and the proposed Bi-Directional Task Adaptation (BDTA) modules are integrated in parallel to form a task bypass branch that extracts adaptation features required by multiple specific tasks. This adaptation module is lightweight, efficient, and does not require backpropagation through the large pre-trained backbone, thus avoiding resource-intensive gradient computations. Moreover, a Mixture of Task Experts mechanism (MoTE) is further proposed to integrate adaptation features across tasks and scales, thereby obtaining more robust representations tailored for dense prediction tasks. On the PASCAL-Context benchmark, our method achieves over 2× relative performance improvement compared to the best prior multi-task PEFT method, while using only ~30% of the parameters, ~50% of the memory, and ~60% of the training time, demonstrating superior overall adaptation efficiency.

PDF Details DOI

AAAI Conference 2026 Conference Paper

ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration

Fanpu Cao
Yaofo Chen
Zeng You
Wei Luo

Diffusion Transformers (DiTs) have achieved state-of-the-art performance in generative modeling, yet their high computational cost hinders real-time deployment. While feature caching offers a promising training-free acceleration solution by exploiting temporal redundancy, existing methods suffer from two key limitations: (1) uniform caching intervals fail to align with the non-uniform temporal dynamics of DiT, and (2) naive feature reuse with excessively large caching intervals can lead to severe error accumulation. In this work, we analyze the evolution of DiT features during denoising and reveal that both feature changes and error propagation are highly time- and depth-varying. Motivated by this, we propose ProCache, a training-free dynamic feature caching framework that addresses these issues via two core components: (i) a constraint-aware caching pattern search module that generates non-uniform activation schedules through offline constrained sampling, tailored to the model’s temporal characteristics; and (ii) a selective computation module that selectively compute within deep blocks and high-importance tokens for cached segments to mitigate error accumulation with minimal overhead. Extensive experiments on PixArt-alpha and DiT demonstrate that ProCache achieves up to 1.96 times and 2.90 times acceleration with negligible quality degradation, significantly outperforming prior caching-based methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

TDSS: Task Dynamic-Synergistic Skill Adaptation for Boosting Efficient and Scalable Multi-Task Learning in Dense Visual Prediction

Haiming Yao
Qiyu Chen
Wei Luo
Zheng Zhang
Jianxing Liao
Wei You

The transfer of knowledge from large-scale pre-trained models to diverse downstream tasks has achieved remarkable success. Beyond the traditional full fine-tuning paradigm, Parameter-Efficient Fine-Tuning (PEFT) has emerged as a more efficient model adaptation approach. However, applying existing PEFT methods to adapt dense vision models, particularly in multi-task settings, remains inadequately explored due to their low efficiency, limited task scalability, and neglect of cross-task fine-tuning interactions. To address these challenges, we propose the Task Dynamic-Synergistic Skill Adaptation, termed TDSS, an efficient and scalable multi-task model adaptation framework for dense visual predictions. TDSS comprises two key components: Task-Dynamic Skill Adapters (TDSA) and Task-Synergistic Adaptation Interaction (TSAI). Specifically, TDSA are inserted in parallel into pre-trained vision models to extract task-specific adapted features through the construction of skill representation experts and task dynamic gating. TSAI is developed to enhance cross-task adaptation interaction by bridging global generic and task-specific adapted features. Extensive experiments on multi-task dense visual predictions demonstrate that TDSS surpasses existing state-of-the-art parameter-efficient fine-tuning methods, while exhibiting remarkable efficiency and scalability in parameters and computational complexity.

PDF Details DOI

AAAI Conference 2026 Conference Paper

WikiMAG: A Multi-Agent Guided Framework for Generating Structured Wikipedia-like Articles

Xiuli Kang
Yinlong Xiao
Minghao Hu
Yuan Huang
Bin Mao
Ming Wang
Fang Wang
Zhunchen Luo

Wikipedia serves as the world's largest and most popular online reference encyclopedia, rich in structured knowledge and authoritative citations. Recently, numerous works have leveraged large language models to automatically generate Wikipedia-like articles. However, existing approaches primarily focus on producing singular narrative-type content, overlooking higher information-density structured elements such as timeline and table. To address these limitations, we propose WikiMAG, a multi-agent guided framework for generating structured Wikipedia-like articles. This framework employs a collaborative multi-agent mechanism to orchestrate the creation process, featuring three synergistic core components: Progressive planner first constructs the coarse-grained outline framework and then annotate fine-grained types for outline units, encompassing narrative, timeline, and table formats; Reflective inspector dynamically curates high-quality references via multi-round interactive feedback, thereby enhancing the authority and relevance of citations; Versatile writer integrates fine-grained outline details and high-quality reference information to generate information-rich articles, incorporating the three annotated formats. We evaluate WikiMAG on two public datasets, FreshWiki and WikiGenBen, across outline, writing, and verifiability dimensions. Compared with the best baseline method, our method achieves an average improvement of 6.73 points and 4.39 points in Heading Soft Recall and the METEOR metric (a machine translation and text generation evaluation metric) respectively, and an average increase of 16.84 percentage points in Citation Rate.

PDF Details DOI

EAAI Journal 2025 Journal Article

Adversarial contrastive domain-generative learning for bacteria Raman spectrum joint denoising and cross-domain identification

Haiming Yao
Wei Luo
Xue Wang

Raman spectroscopy, as a label-free detection technology, has been widely utilized in the clinical diagnosis of pathogenic bacteria. However, Raman signals are naturally weak and sensitive to the condition of the acquisition process. The characteristic spectra of a bacteria can manifest varying signal-to-noise ratios and domain discrepancies under different acquisition conditions. Consequently, existing methods often face challenges when identifying unobserved acquisition conditions, i. e. , the testing acquisition conditions are unavailable during model training. In this article, a generic framework, namely, an adversarial contrastive domain-generative learning framework, is proposed for joint Raman spectroscopy denoising and cross-domain identification. The proposed method is composed of a domain generation module and a domain task module. Through adversarial learning between these two modules, it utilizes only a single available source domain spectral data to generate extended denoised domains that are semantically consistent with the source domain and extract domain-invariant representations. Experimental results show that the proposed method significantly enhances diagnostic performance, with an average recognition accuracy improvement of over +5. 0% under unknown acquisition conditions compared to existing methods. Notably, the proposed method also performs simultaneous denoising of the spectra, enhancing the signal-to-noise ratio of the original signal by an average gain of +3. 0, without requiring noise-free ground truth. These results suggest that the proposed method holds great potential as a diagnostic tool for real-world clinical cases.

Details DOI

IJCAI Conference 2025 Conference Paper

Automated Detection of Pre-training Text in Black-box LLMs

Ruihan Hu
Yu-Ming Shang
Jiankun Peng
Wei Luo
Yazhe Wang
Xi Zhang

Detecting whether a given text is a member in the pre-training data of Large Language Models (LLMs) is crucial for ensuring data privacy and copyright protection. Most existing methods rely on the LLM's hidden information (e. g. , model parameters or token probabilities), making them ineffective in the black-box setting, where only input and output texts are accessible. Although some methods have been proposed for the black-box setting, they rely on massive manual efforts such as designing complicated questions or instructions. To address these issues, we propose VeilProbe, the first framework for automatically detecting LLMs' pre-training texts in a black-box setting without human intervention. VeilProbe utilizes a sequence-to-sequence mapping model to infer the latent mapping feature between the input text and the corresponding output suffix generated by the LLM. Then it performs the key token perturbations to obtain more distinguishable membership features. Additionally, considering real-world scenarios where the ground-truth training text samples are limited, a prototype-based membership classifier is introduced to alleviate the overfitting issue. Extensive evaluations on three widely used datasets demonstrate that our framework is effective and superior in the black-box setting.

PDF Details DOI

AAAI Conference 2025 Conference Paper

ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing

Yaohui Ma
Xiaopeng Hong
Shizhou Zhang
Huiyun Li
Zhilin Zhu
Wei Luo
Zhiheng Ma

Large multimodal language models (MLLMs) have revolutionized natural language processing and visual understanding, but often contain outdated or inaccurate information. Current multimodal knowledge editing evaluations are limited in scope and potentially biased, focusing on narrow tasks and failing to assess the impact on in-domain samples. To address these issues, we introduce ComprehendEdit, a comprehensive benchmark comprising eight diverse tasks from multiple datasets. We propose two novel metrics: Knowledge Generalization Index (KGI) and Knowledge Preservation Index (KPI), which evaluate editing effects on in-domain samples without relying on AI-synthetic samples. Based on insights from our framework, we establish Hierarchical In-Context Editing (HICE), a baseline method employing a two-stage approach that balances performance across all metrics. This study provides a more comprehensive evaluation framework for multimodal knowledge editing, reveals unique challenges in this field, and offers a baseline method demonstrating improved performance. Our work opens new perspectives for future research and provides a foundation for developing more robust and effective editing techniques for MLLMs.

PDF Details DOI

EAAI Journal 2025 Journal Article

Feature transformation and statistical calibration for cross-domain few-shot classification

Jiafan Liu
Jin Deng
Jinrong Cui
Wei Luo

The limited labeled training samples in target tasks, combined with the distribution discrepancy between the target and the model-learning source datasets, create a significant challenge in Remote Sensing Scene Classification (RSSC). Existing work attacks this problem from the perspective of feature generalizability while ignoring model bias. In this paper, we address this issue in the framework of Cross-Domain Few-Shot Classification (CDFSC) and propose an overfitting-resistant and debiased learning method, FEature Transformation and stAtistical Calibration (FETAC), to tackle this problem. In FETAC, linear and residual modules are incorporated to the backbone to transform features layer-wisely for model overfitting-resistant learning at the task level. Meanwhile, a parameter-free Lifted Laplacian Activation (LLA) unit is proposed to encourage balanced activations of features across tasks on the target domain at the dataset level. This property of balanced activations prevents the model from using only a small number of features for adaptation, thus achieving feature calibration statistically for an unbiased cross-domain feature transformation. In model architecture, FETAC is designed as a plug-and-play module on the backbone and is optimized on the target few-shot task at test time on the fly. Experiments on five distinct RSSC target domains validate the superior performance of FETAC for CDFSC tasks. Furthermore, the excellent performance exhibited by FETAC on the 1-shot CDFSC tasks further demonstrates its robust applicability in RSSC under extremely scarce labeled examples. Code will be available at https: //github. com/JiaHg/fetac.

Details DOI

JBHI Journal 2025 Journal Article

Learning to Detect Sleep Micro-Events from Coarse Sleep Stage Annotations

Chenhao Wang
Yan Pei
Jing Hu
Chengyang Han
Jiahui Xu
Lisan Zhang
Feng Yu
Bo Jin

Sleep micro-events, such as sleep spindles and K-complexes, are closely associated with neurological cognitive functions. While artificial intelligence (AI)-assisted sleep micro-event detection provides automated annotation to reduce reliance on labor-intensive expert labeling, current supervised approaches require precisely annotated datasets that remain scarce in clinical practice. To overcome this data bottleneck, this paper introduces a Weakly Supervised Sleep Micro-Event Detector (WSSMED) that leverages readily available coarse sleep stage annotations. The proposed WSSMED features a dual-branch architecture, consisting of a wave prototype module and a cluster module, designed to capture the fine-grained sleep micro-event patterns experts rely on for sleep staging. This framework infers expert logic from coarse annotations while mitigating performance degradation caused by annotation inconsistencies arising from inter-rater variability. Experiments conducted on two public datasets and one clinical dataset demonstrate that WSSMED achieves state-of-the-art performance in detecting sleep spindles and K-complexes, as evaluated at both sample-level and event-level in terms of precision, recall and F1-score metrics. Furthermore, subject-level evaluation demonstrates that the density and duration of micro-events detected by WSSMED-key metrics linked to cognitive function and neurological status-align more closely with expert annotations than those of other reported methods. These results highlight the clinical potential of WSSMED for reliable sleep micro-event analysis.

Details DOI

JBHI Journal 2025 Journal Article

Swallow-PPG: Photoplethysmography Templates for Comprehensive Temporal Analysis of Swallowing Anatomical Actions

Ying Zhang
Junjie Li
Ping Wang
Huaiyu Zhu
Bo Wang
Wei Luo
Yun Pan

In clinical practice, Videofluoroscopic Swallowing Study (VFSS) is commonly used to monitor the activity of anatomical structures during swallowing. However, it is limited by ionizing radiation exposure, adverse effects of barium contrast agents, and the high cost of specialized equipment. In this study, we propose a framework for analyzing swallowing behaviors in photoplethysmography (PPG) waveforms, which includes generalizing the manifestation of swallowing in PPG (i. e. , swallowing templates generation) and conducting comprehensive temporal analysis of swallowing anatomical actions (TASAA). For swallowing templates generation, we cluster and average the samples to obtain waveforms of templates, followed by conducting shape-based mapping and averaging on 28 time indicators to derive template unified time indicators (TUTIs). For comprehensive TASAA, we leverage templates waveforms and TUTIs to estimate time indicators based on the mapping relationship between samples and their respective templates. We evaluate the proposed framework on 357 swallowing PPG samples from 41 elderly subjects. The average relative error across all time indicators is 0. 123, and 6 indicators notably excel with errors below 0. 1. The proposed template-based swallowing analysis framework is expected to become a low-cost and non-ionizing alternative to VFSS for comprehensive TASAA.

Details DOI

JBHI Journal 2025 Journal Article

WaveSleepNet: An Interpretable Network for Expert-Like Sleep Staging

Yan Pei
Jiahui Xu
Feng Yu
Lisan Zhang
Wei Luo

Although deep learning algorithms have proven their efficiency in automatic sleep staging, their “black-box” nature has limited their clinical adoption. In this study, we propose WaveSleepNet, an interpretable neural network for sleep staging that reasons in a similar way to sleep clinical experts. In this network, we utilize the latent space representations generated during training to identify characteristic wave prototypes corresponding to different sleep stages. The feature representation of an input signal is segmented into patches within the latent space, each of which is compared against the learned wave prototypes. The proximity between these patches and the wave prototypes is quantified through scores, indicating the prototypes' presence and relative proportion within the signal. The scores serve as the decision-making criteria for final sleep staging. During training, an ensemble of loss functions is employed for the prototypes' diversity and robustness. Furthermore, the learned wave prototypes are visualized by analyzing occlusion sensitivity. The efficacy of WaveSleepNet is validated across three public datasets, achieving sleep staging performance that are on par with those of the state-of-the-art models. A detailed case study examining the decision-making process of WaveSleepNet demonstrates that it aligns closely with American Academy of Sleep Medicine (AASM) manual guidelines. Another case study systematically explained the misidentified reasons behind each sleep stage. WaveSleepNet's transparent process provides specialists with direct access to the physiological significance of the model's criteria, allowing for future validation, adoption and further enrichment by sleep clinical experts.

Details DOI

JMLR Journal 2024 Journal Article

A Semi-parametric Estimation of Personalized Dose-response Function Using Instrumental Variables

Wei Luo
Yeying Zhu
Xuekui Zhang
Lin Lin

In the application of instrumental variable analysis that conducts causal inference in the presence of unmeasured confounding, invalid instrumental variables and weak instrumental variables often exist which complicate the analysis. In this paper, we propose a model-free dimension reduction procedure to select the invalid instrumental variables and refine them into lower-dimensional linear combinations. The procedure also combines the weak instrumental variables into a few stronger instrumental variables that best condense their information. We then introduce the personalized dose-response function that incorporates the subject's personal characteristics into the conventional dose-response function, and use the reduced data from dimension reduction to propose a novel and easily implementable nonparametric estimator of this function. The proposed approach is suitable for both discrete and continuous treatment variables, and is robust to the dimensionality of data. Its effectiveness is illustrated by the simulation studies and the data analysis of ADNI-DoD study, where the causal relationship between depression and dementia is investigated. [abs] [ pdf ][ bib ] &copy JMLR 2024. ( edit, beta )

PDF Details

AAAI Conference 2024 Conference Paper

Divergence-Guided Simultaneous Speech Translation

Xinjie Chen
Kai Fan
Wei Luo
Linlin Zhang
Libo Zhao
Xinggao Liu
Zhongqiang Huang

To achieve high-quality translation with low latency, a Simultaneous Speech Translation (SimulST) system relies on a policy module to decide whether to translate immediately or wait for additional streaming input, along with a translation model capable of effectively handling partial speech input. Prior research has tackled these components separately, either using ``wait-k'' policies based on fixed-length segments or detected word boundaries, or dynamic policies based on different strategies (e.g., meaningful units), while employing offline models for prefix-to-prefix translation. In this paper, we propose Divergence-Guided Simultaneous Speech Translation (DiG-SST), a tightly integrated approach focusing on both translation quality and latency for streaming input. Specifically, we introduce a simple yet effective prefix-based strategy for training translation models with partial speech input, and develop an adaptive policy that makes read/write decisions for the translation model based on the expected divergence in translation distributions resulting from future input. Our experiments on multiple translation directions of the MuST-C benchmark demonstrate that our approach achieves a better trade-off between translation quality and latency compared to existing methods.

PDF Details DOI

JBHI Journal 2024 Journal Article

DTP-Net: Learning to Reconstruct EEG Signals in Time-Frequency Domain by Multi-Scale Feature Reuse

Yan Pei
Jiahui Xu
Qianhao Chen
Chenhao Wang
Feng Yu
Lisan Zhang
Wei Luo

Electroencephalography (EEG) signals are prone to contamination by noise, such as ocular and muscle artifacts. Minimizing these artifacts is crucial for EEG-based downstream applications like disease diagnosis and brain-computer interface (BCI). This paper presents a new EEG denoising model, DTP-Net. It is a fully convolutional neural network comprising Densely-connected Temporal Pyramids (DTPs) placed between two learnable time-frequency transformations. In the time-frequency domain, DTPs facilitate efficient propagation of multi-scale features extracted from EEG signals of any length, leading to effective noise reduction. Comprehensive experiments on two public semi-simulated datasets demonstrate that the proposed DTP-Net consistently outperforms existing state-of-the-art methods on metrics including relative root mean square error (RRMSE) and signal-to-noise ratio improvement ( $\Delta$ SNR). Moreover, the proposed DTP-Net is applied to a BCI classification task, yielding an improvement of up to 5. 55% in accuracy. This confirms the potential of DTP-Net for applications in the fields of EEG-based neuroscience and neuro-engineering. An in-depth analysis further illustrates the representation learning behavior of each module in DTP-Net, demonstrating its robustness and reliability.

Details DOI

EAAI Journal 2024 Journal Article

Local–global normality learning and discrepancy normalizing flow for unsupervised image anomaly detection

Haiming Yao
Wei Luo
Weihang Zhang
Xiaotian Zhang
Zhenfeng Qiang
Donghao Luo

The unsupervised detection and localization of image anomalies hold significant importance across various domains, particularly in industrial quality inspection. Despite its widespread utilization, this task remains inherently challenging due to its reliance solely on defect-free normal knowledge. This paper presents the local–global normality learning and discrepancy normalizing flow, a new state-of-the-art model for unsupervised image anomaly detection and localization. In contrast to existing methods, It adopts a two-stream approach that considers both local and global semantics, ensuring stable detection of abnormalities. The framework comprises two key components: the dual-branch Transformer and the discrepancy normalizing flow, facilitating reconstruction and discrimination. The proposed framework leverages pre-trained convolutional neural networks to extract multi-scale feature embeddings, followed by a novel dual-branch transformer that achieves feature reconstruction from local and global perspectives. The local reconstruction employs self-attention, while the global reconstruction incorporates global prototype tokens and semantic query tokens by the aggregation-cross attention mechanism. Moreover, discrepancy normalizing flow is developed to estimate the likelihood of anomalies based on the discrepancy between pre-trained features and local/global reconstruction results. Extensive validation on established public benchmarks confirms that our method achieves state-of-the-art performance with the proposed local–global reconstruction and discrimination dual-stream framework.

Details DOI

ECAI Conference 2024 Conference Paper

MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos

Yin Tang
Wei Luo
Jinrui Zhang
Wei Huang
Ruihai Jing
Deyu Zhang

Multi-instance Repetitive Action Counting (MRAC) aims to estimate the number of repetitive actions performed by multiple instances in untrimmed videos, commonly found in human-centric domains like sports and exercise. In this paper, we propose MultiCounter, a fully end-to-end deep learning framework that enables simultaneous detection, tracking, and counting of repetitive actions of multiple human instances. Specifically, MultiCounter incorporates two novel modules: 1) mixed spatiotemporal interaction for efficient context correlation across consecutive frames, and 2) task-specific heads for accurate perception of periodic boundaries and generalization for action-agnostic human instances. We train MultiCounter on a synthetic dataset called MultiRep that is generated from annotated real-world videos. Experiments on the MultiRep dataset validate the fundamental challenge of MRAC tasks and showcase the superiority of our proposed model. Compared to ByteTrack+RepNet, a solution that combines an advanced tracker with a single repetition counter, MultiCounter substantially improves Period-mAP by 41. 0%, reduces AvgMAE by 58. 6%, and increases AvgOBO 1. 48 times. This sets a new benchmark in the field of MRAC. Moreover, MultiCounter runs in real-time on a commodity GPU server and is insensitive to the number of human instances in a video.

Details

EAAI Journal 2024 Journal Article

Template-based Feature Aggregation Network for industrial anomaly detection

Wei Luo
Haiming Yao
Wenyong Yu

Industrial anomaly detection plays a crucial role in ensuring product quality control. Therefore, proposing an effective anomaly detection model is of great significance. While existing feature-reconstruction methods have demonstrated excellent performance, they face challenges with shortcut learning, which can lead to undesirable reconstruction of anomalous features. To address this concern, we present a novel feature-reconstruction model called the Template-based Feature Aggregation Network (TFA-Net) for anomaly detection via template-based feature aggregation. Specifically, TFA-Net first extracts multiple hierarchical features from a pre-trained convolutional neural network for a fixed template image and an input image. Instead of directly reconstructing input features, TFA-Net aggregates them onto the template features, effectively filtering out anomalous features that exhibit low similarity to normal template features. Next, TFA-Net utilizes the template features that have already fused normal features in the input features to refine feature details and obtain the reconstructed feature map. Finally, the defective regions can be located by comparing the differences between the input and reconstructed features. Our template-based feature aggregation schema yields a nontrivial and meaningful feature reconstruction task. The simple, yet efficient, TFA-Net exhibits state-of-the-art detection performance on various real-world industrial datasets. Additionally, it fulfills the real-time demands of industrial scenarios, rendering it highly suitable for practical applications in the industry.

Details DOI

IJCAI Conference 2023 Conference Paper

A Canonicalization-Enhanced Known Fact-Aware Framework For Open Knowledge Graph Link Prediction

Yilin Wang
Minghao Hu
Zhen Huang
Dongsheng Li
Wei Luo
Dong Yang
Xicheng Lu

Open knowledge graph (OpenKG) link prediction aims to predict missing factual triples in the form of (head noun phrase, relation phrase, tail noun phrase). Since triples are not canonicalized, previous methods either focus on canonicalizing noun phrases (NPs) to reduce graph sparsity, or utilize textual forms to improve type compatibility. However, they neglect to canonicalize relation phrases (RPs) and triples, making OpenKG maintain high sparsity and impeding the performance. To address the above issues, we propose a Canonicalization-Enhanced Known Fact-Aware (CEKFA) framework that boosts link prediction performance through sparsity reduction of RPs and triples. First, we propose a similarity-driven RP canonicalization method to reduce RPs' sparsity by sharing knowledge of semantically similar ones. Second, to reduce the sparsity of triples, a known fact-aware triple canonicalization method is designed to retrieve relevant known facts from training data. Finally, these two types of canonical information are integrated into a general two-stage re-ranking framework that can be applied to most existing knowledge graph embedding methods. Experiment results on two OpenKG datasets, ReVerb20K and ReVerb45K, show that our approach achieves state-of-the-art results. Extensive experimental analyses illustrate the effectiveness and generalization ability of the proposed framework.

PDF Details DOI

EAAI Journal 2023 Journal Article

A two-stream network with complementary feature fusion for pest image classification

Chao Wang
Jinrui Zhang
Jin He
Wei Luo
Xiaohui Yuan
Lichuan Gu

Pests are diverse and the available datasets often contain an uneven number of examples for different pests (a. k. a. , the long-tail distribution). This poses a great challenge to learning-based classification methods, especially deep networks, and often leads to degraded performance, especially for the minority (tail) classes. This paper presents a deep learning integration architecture based on decoupling training and fusion learning, which integrates different models with complementary performance on pest datasets with a long-tailed distribution to improve the overall classification performance of pests. A deep neural network is designed that fuses two complementary deep learning models at the feature level, which consists of a convolution neural network (ConvNeXt) and a Swin Transformer model for decoupling training. Experiments are conducted using three datasets (d0, insect, and IP102), and evaluation on accuracy, recall, and F1-Score is reported. For the large-scale pest dataset with long-tailed distribution IP102, the accuracy achieves 76. 1%, which outperforms the state-of-the-art methods. In addition, the accuracy for d0 and insect datasets are 98. 5% and 92. 3%, respectively.

Details DOI

IJCAI Conference 2023 Conference Paper

Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?

Bing Liu
Wei Luo
Gang Li
Jing Huang
Bo Yang

As deep learning gains popularity in modelling dynamical systems, we expose an underappreciated misunderstanding relevant to modelling dynamics on networks. Strongly influenced by graph neural networks, latent vertex embeddings are naturally adopted in many neural dynamical network models. However, we show that embeddings tend to induce a model that fits observations well but simultaneously has incorrect dynamical behaviours. Recognising that previous studies narrowly focus on short-term predictions during the transient phase of a flow, we propose three tests for correct long-term behaviour, and illustrate how an embedding-based dynamical model fails these tests, and analyse the causes, particularly through the lens of topological conjugacy. In doing so, we show that the difficulties can be avoided by not using embedding. We propose a simple embedding-free alternative based on parametrising two additive vector-field components. Through extensive experiments, we verify that the proposed model can reliably recover a broad class of dynamics on different network topologies from time series data.

PDF Details DOI

AAAI Conference 2019 Conference Paper

Robust Anomaly Detection in Videos Using Multilevel Representations

Hung Vu
Tu Dinh Nguyen
Trung Le
Wei Luo
Dinh Phung

Detecting anomalies in surveillance videos has long been an important but unsolved problem. In particular, many existing solutions are overly sensitive to (often ephemeral) visual artifacts in the raw video data, resulting in false positives and fragmented detection regions. To overcome such sensitivity and to capture true anomalies with semantic significance, one natural idea is to seek validation from abstract representations of the videos. This paper introduces a framework of robust anomaly detection using multilevel representations of both intensity and motion data. The framework consists of three main components: 1) representation learning using Denoising Autoencoders, 2) level-wise representation generation using Conditional Generative Adversarial Networks, and 3) consolidating anomalous regions detected at each representation level. Our proposed multilevel detector shows a significant improvement in pixel-level Equal Error Rate, namely 11. 35%, 12. 32% and 4. 31% improvement in UCSD Ped 1, UCSD Ped 2 and Avenue datasets respectively. In addition, the model allowed us to detect mislabeled anomalies in the UCDS Ped 1.

PDF Details

I&C Journal 2010 Journal Article

Mind change optimal learning of Bayes net structure from dependency and independency data

Oliver Schulte
Wei Luo
Russell Greiner

This paper analyzes the problem of learning the structure of a Bayes net in the theoretical framework of Gold’s learning paradigm. Bayes nets are one of the most prominent formalisms for knowledge representation and probabilistic and causal reasoning. We follow constraint-based approaches to learning Bayes net structure, where learning is based on observed conditional dependencies and independencies between variables of interest (e. g. , the data are of the form “X is dependent on Y given any assignment to variables S” or of the form “X is independent of Y given any assignment to variables S”). Applying learning criteria in this model leads to the following results. (1) The mind change complexity of identifying a Bayes net graph over variables V from either dependency data or from independency data are | v | 2, the maximum number of edges. (2) There is a unique fastest mind-change optimal Bayes net learner for either data type; convergence speed is evaluated using Gold’s dominance notion of “uniformly faster convergence”. For dependency data, the optimal learner conjectures a graph if it is the unique Bayes net pattern that satisfies the observed dependencies with a minimum number of edges, and outputs “no guess” otherwise. For independency data, the optimal learner conjectures a graph if it is the unique Bayes net pattern that satisfies the observed dependencies with a maximum number of edges, and outputs “no guess” otherwise. We investigate the complexity of computing the output of the fastest mind-change optimal learner for either data type, and show that each of these two problems is NP-hard (assuming P=RP). To our knowledge these are the first NP-hardness results concerning the existence of a uniquely optimal Bayes net structure.

Details DOI

I&C Journal 2006 Journal Article

Mind change efficient learning

Wei Luo
Oliver Schulte

This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of strong mind change optimality. We characterize the structure of language classes that can be identified with at most α mind changes by some learner (not necessarily effective): a language class L is identifiable with α mind changes iff the accumulation order of L is at most α. Accumulation order is a classic concept from point-set topology. We show that accumulation order is related to other established notions of structural complexity, such as thickness and intrinsic complexity. To aid the construction of learning algorithms, we show that the characteristic property of strongly mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. We illustrate the theory by describing strongly mind change optimal learners for various problems such as identifying linear subspaces, one-variable patterns, and fixed-length patterns.

Details DOI