Qing Li

AAAI Conference 2026 Conference Paper

AIM: Manifold-based Data Filtering for Representation Finetuning

• Qing Li
• Qibin Zheng
• Yi Liu
• Xingchun Diao

Representation Finetuning (ReFT) has recently emerged as an efficient paradigm for adapting pretrained language models by editing hidden representations rather than model weights. However, our preliminary experiments reveal that ReFT is notably more sensitive to training data quality compared to traditional parameter-efficient finetuning methods, particularly to samples with incorrect labels, which can severely degrade performance. Inspired by prior work demonstrating that the hidden representations of generalizable neural networks exhibit low-dimensional manifold structures, we hypothesize that effective generalization in ReFT requires geometrically structured transformations between pre- and post-intervention representations. This implies that the intervention vectors representing these transformations should form a low-dimensional manifold, rendering the inconsistent transformations induced by label noise as detectable geometric outliers. To leverage this insight, we introduce Aligning Interventions on a learned Manifold (AIM), a representation-based data filtering method for ReFT, which identifies high-quality training samples by measuring the geometric consistency of their intervention vectors with respect to a robust reference manifold derived via principal component analysis on trusted data. Extensive experiments on both commonsense and arithmetic reasoning tasks confirm the effectiveness of AIM, showing consistent improvements over strong data selection baselines across multiple model scales.

EAAI Journal 2026 Journal Article

BoA-SQL: Executable Blueprint-of-Action for Text-to-SQL with reinforcement learning

• Yang Wang
• Zhilong Xie
• Lin Zhang
• Lingyun Gu
• Qing Li

Recent progress in large language models has opened new possibilities for querying databases using natural language instead of SQL. Yet existing methods, often relying on linear reasoning, struggle with complex nested logic and are susceptible to error propagation. We propose BoA-SQL, which turns reasoning into a durable, structured plan. Our contributions are threefold: (1) To resolve schema ambiguity and context overload, a lightweight, knowledge graph-driven linker grounds the query by pruning irrelevant schema before planning. (2) To overcome the structural mismatch of linear plans, a persistent, tree-structured blueprint aligns with SQL’s hierarchy, enabling localized repair of faulty segments without full re-computation. (3) To align the language model’s text-generation objective with execution correctness, a two-stage reinforcement learning policy optimizes the entire blueprint for task success. Extensive evaluations on public benchmarks Spider and a complex real-world database validate this approach. BoA-SQL achieves 85. 6% execution accuracy. This figure notably increases to 88. 2% after correcting benchmark label errors. These findings suggest that durable, structure-aware planning, combined with schema grounding and planning-aligned optimization, is a practical path to reliable Text-to-SQL.

AAAI Conference 2026 Conference Paper

CPOStream: Collaborating Prediction and Observation for Flicker-Free Streamable Free-Viewpoint Video with 3DGS

• Zhenyu Bao
• Qing Li
• Jinhan Xie
• Kanglin Liu

3D Gaussian Splatting (3DGS) has recently demonstrated significant potential for streaming dynamic scenes, enabling the synthesis of photo-realistic and real-time free-viewpoint videos (FVVs). Conventional streaming pipelines optimize each frame independently, i.e., the attribute of the 3D Gaussians (3DGs) responsible for the static regions are supposed to be identical across all frames but are changed in the optimization process, thus causing temporal color inconsistency and visual flickering artifacts in the static regions. To tackle this, we propose CPOStream, which utilizes a prediction and observation module to determine the state of 3DG. Specifically, the prediction module records those 3DGs that are inactive in the past K frames and those would be ignored in the optimization process of the current frame reconstruction. Thus, the attributes of those 3DGs would be kept consistent across the past K frames, guaranteeing the temporal consistence. Additionally, the observation module conducts motion detection, and recognizes those new 3DGs which are not recorded in the prediction module and are first detected by the observation module in the past K frames. The attributes of those 3DGs are optimized during the current frame reconstruction. Experiments on multiple real-world FVV benchmarks show that CPOStream substantially reduces temporal flickering and improves reconstruction fidelity, achieving state‑of‑the‑art performance.

JBHI Journal 2026 Journal Article

GAEA-Net: Generating Activity-Enriched Abnormal ECGs via Adversarial Network

• Liuqing Chen
• Shuhong Xiao
• Yujie Zang
• Jiner Wang
• Shanhai Hu
• Lin Zhang
• Qing Li
• Danyang Zhang

With the increasing demand for personalized health monitoring through wearable devices, there is a growing need for non-prescription ECG diagnosing, especially during physical activities. However, existing abnormal ECG data are typically measured in clinical settings, reflecting heart waveforms in a resting state. Abnormality classification models based on such data often struggle to maintain high performance during physical activities, leading to increased false alarms and a higher incidence of missed detections. Due to the potential risks associated with having patients engage in physical activity, abnormal ECG data captured during exercise is not readily available, further complicating the development of reliable models for active scenarios. To address this issue, we propose GAEA-Net in this study. Our goal is to utilize exercise ECGs from healthy individuals, which are more easily accessible, combined with resting-state abnormal ECGs, to generate activity-enriched ECGs through synthesis. We conduct abnormal classification on five widely used datasets, achieving average improvements of 1. 3% in Accuracy, 1. 3% in F1-score, 0. 9% in AUROC, 1. 6% in MCC, and 1. 4% in Cohen’s Kappa. Furthermore, a clinical Turing test involving seven experienced cardiologists confirms that our synthesized ECGs exhibit high fidelity. In the diagnostic task, the cardiologists achieved comparable accuracy on synthetic and real ECGs (55. 7% vs. 54. 9%, p = 0. 76).

AAAI Conference 2026 Conference Paper

RatioSketch: Towards More Accurate Frequency Estimation in Data Streams via a Lightweight Neural Network

• Mengbo Wang
• Zhuochen Fan
• Dayu Wang
• Guorui Xie
• Qing Li
• Zeyu Luan
• Yong Jiang
• Tong Yang

Sketch-based solutions are widely used to estimate item frequencies in infinite data streams.Traditional hand-crafted sketches face the bottleneck of further eliminating errors because they cannot fully utilize the data stream distribution.Although recent neural sketches represented by MetaSketch and LegoSketch have improved generalization capabilities, they face bottlenecks such as high computational overhead and parameter sensitivity.Meanwhile, they ignore load information, fail to fully utilize the local information in hand-crafted sketches, and do not focus on the frequent items that are usually more important in data streams.In this paper, we propose RatioSketch, a novel lightweight neural network correction framework that synergizes the advantages of hand-crafted sketches and neural sketches in a ``micro-correction'' paradigm.The key idea is to retain the efficient underlying data structure of the hand-crafted sketch and to build a neural correction layer in its output space. We select multiple representative hand-crafted sketches as use cases to study the correction performance of RatioSketch on them.Extensive experimental evaluations on several real-world datasets show that RatioSketch-corrected sketches achieve consistently higher estimation accuracy than their uncorrected counterparts, as well as outperforming neural baselines such as MetaSketch and LegoSketch under identical memory budgets.

AAAI Conference 2026 Conference Paper

Suit the Remedy to the Retriever: Interpretable Query Optimization with Retriever Preference Alignment for Vision-Language Retrieval

• GuangHao Meng
• Jinpeng Wang
• Jieming Zhu
• Letian Zhang
• Yong Jiang
• Dan Zhao
• Qing Li

Vision-language retrieval (VLR), which uses text or image queries to retrieve corresponding cross-modal content, plays a crucial role in multimedia and computer vision tasks. However, challenging concepts in queries often confuse retrievers, limiting their ability to align concepts with visual content. Existing query optimization methods neglect retrievers’ preferences (i.e., text descriptions that better match their corresponding visual content), resulting in unadapted to the retriever and leading to suboptimal performance. To address this, we propose the Retriever-Adaptive Query Optimization (RAQO), an interpretable framework that rewrites queries based on retriever-specific preferences. Specifically, we first leverages multimodal large language Models (MLLMs) and retrieval's feedback to construct the MLLMs-Driven Preference-Aware Dataset Engine (MPADE), which automatically refine queries offline, capturing the retriever’s implicit preferences. Then, we introduce a ``detect-then-rewrite" chain-of-thought rewriting (ReCoT) strategy equipped with a progressive preference alignment pipeline, including three stages: ambiguity detection fine-tuning, query rewriting fine-tuning, and preference rank optimization. This design enables the rewriter to focus on confusing concepts and produce retriever-adapted, high-quality queries. Extensive VLR benchmark experiments have demonstrated the superiority of RAQO in cross-modal retrieval, as well as its interpretability, generalizability and transferability.

AAAI Conference 2026 Conference Paper

TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents

• Bofei Zhang
• Zirui Shang
• Zhi Gao
• Wang Zhang
• Rui Xie
• Xiaojian Ma
• Tao Yuan
• Xinxiao Wu

Building Graphical User Interface (GUI) agents is a promising research direction, which simulates human interaction with computers or mobile phones to perform diverse GUI tasks. However, a major challenge in developing generalized GUI agents is the lack of sufficient trajectory data across various operating systems and applications, mainly due to the high cost of manual annotations. In this paper, we propose the TongUI framework that transforms millions of multimodal web tutorials into GUI trajectories for generalized GUI agents. Concretely, we crawl GUI videos and articles from the Internet and process them into GUI agent trajectory data. Based on this, we construct the GUI-Net-1M dataset, which contains 1 million trajectories across five operating systems and over 280 applications. To the best of our knowledge, this is the largest open-source GUI trajectory dataset. We develop the TongUI agent by fine-tuning Qwen2.5-VL-3B/7B/32B models on GUI-Net-1M, which shows consistent performance improvements on commonly used grounding and navigation benchmarks, outperforming baseline agents by 10\% on multiple benchmarks, showing the effectiveness of the GUI-Net-1M dataset and underscoring the significance of our TongUI framework.

EAAI Journal 2026 Journal Article

Two level time-aware network for clinical event prediction

• Qing Li
• Zehao Li

Electronic Health Records (EHR) have greatly facilitated predictive models based on deep learning. However, since EHR data contains information features at various levels, existing studies typically focus only on mining information at single level, leading to underutilization of EHR resources and consequently inaccurate predictions of patient conditions. Moreover, existing research often acquires static representations of diseases. In practice, the progression of diseases is dynamic and even the same disease may vary in importance in predicting patient outcomes at different times of visits. To address these issues, we propose a Twolevel, time-aware Clinical Event prediction Model, TCEM. Specifically, we designed a temporal weight subgraph to model the code level structural features and integrate the dynamic impact of visit timing on disease representation. At the visit level, we categorize diseases based on their occurrence or disappearance in adjacent visits to capture information related to disease evolution. We conducted extensive experiments on two real-world datasets, showing that TCEM outperforms existing models in medical event prediction. Our code can be found at https: //github. com/lzh-nwpu/tcem.

EAAI Journal 2025 Journal Article

A latent-coupled neural network for multiphysics long-term forecasting in reactor transients using sparse observations

• Yu-Yan Xu
• Jun Luo
• Deng Pan
• Wei Lu
• Ting Liu
• Guanghui Yuan
• Minxiao Zhong
• Qing Li

Complex dynamical systems in safety-critical applications like nuclear reactors involve strongly coupled physical fields evolving over space and time. Accurate prediction of these fields is vital for safety monitoring but is challenged by limited sensor placement and unobservable variables (e. g. , xenon and iodine concentrations). This paper proposes the Sparse observation to High-dimensional coupled physical field Prediction Network (SHPNet), a deep learning framework that predicts and reconstructs multiple physical fields directly from sparse observations. SHPNet combines a three-branch autoencoder to extract shared latent representations with a neural operator that models temporal dynamics in latent space, enabling efficient long-term forecasting. Evaluated on Hua-long Pressurized Reactor (HPR1000) under varying power and burnup conditions, SHPNet outperforms traditional frameworks and end-to-end model, achieving higher accuracy, robustness to observation sparsity, and effective reconstruction of unobservable fields. These results demonstrate SHPNet’s potential as a practical tool for real-time monitoring of complex coupled systems.

JBHI Journal 2025 Journal Article

A Lightweight Polyp Image Segmentation Model Using Deep Convolution Kernel Modules and Nonlinear Units in Colonoscopy

• Xingchi Chen
• Fushen Xie
• Qing Li
• Fa Zhu
• Kadambri Agarwal

Polyp image segmentation, as one of the important components of medical image processing tasks, provides a powerful auxiliary role for clinical colonoscopy. However, it is very difficult to implement lightweight and high-performance image segmentation models on general platforms, which poses a huge challenge to autonomous AI-driven colonoscopy. In this paper, we propose a lightweight Polyp Image Segmentation Model, including two key components. Firstly, this paper proposes a lightweight deep convolution kernel module (DCKM) to balance segmentation accuracy and inference time. The DCKM with a residual structure connects the input and output so that the output is corrected by the input, thereby improving the segmentation accuracy. Secondly, the proposed DCKM in this paper directly uses a multi-scale convolution structure instead of a complex residual structure to extract local features, reducing the inference time of the proposed model and realizing a lightweight image segmentation model. Finally, we introduce a nonlinear unit (NU) by connecting the convolution block with a nonlinear activation function to realize a nonlinear codec structure to balance the accuracy loss of the model segmentation results caused by the lightweight DCKM. We evaluate the inference time and segmentation performance of the proposed model on four public polyp datasets. Experimental results show that the proposed model reduces the model inference time and obtains better segmentation performance compared with the state-of-the-art methods.

TIST Journal 2025 Journal Article

A Multi-Hop Graph Reasoning Network for Knowledge-Based VQA

• Zihan Hu
• Jiuxiang You
• Zhenguo Yang
• Xiaoping Li
• Haoran Xie
• Qing Li
• Wenyin Liu

Knowledge-based visual question answering (KB-VQA) requires reasoning about the visual grounding relations between the images and questions by incorporating external knowledge. Existing works typically retrieve knowledge from knowledge graphs by leveraging global multimodal representations of image–text pairs for graph convolution, which neglect contextual clues at hop granularity, resulting in suboptimal spreading and leveraging of contextual information. To this end, we propose a multi-hop graph reasoning network (MGRN) for KB-VQA, which consists of a knowledge graph constructor (KGC) module, a semantic-instructed graph reasoning (SGR) module, and an answering module. MGRN exploits multimodal semantics from given images and questions as instructions for graph reasoning to obtain the knowledge representation from either the scene graph or knowledge base. Specifically, KGC fuses the scene graph with triplets from ConceptNet and Comet to construct a contextual knowledge graph for retrieving knowledge representation. Furthermore, SGR conducts multi-hop graph reasoning to select top- K knowledge items for answering by passing and filtering interplay messages on contextual knowledge graphs under the guidance of multimodal semantic representation. Extensive experiments conducted on two public datasets show the effectiveness and outperformance of our method.

EAAI Journal 2025 Journal Article

A novel high-accuracy graph neural network-based rumor detection method

• Xi Xiao
• Zeming Wu
• Chengzong Cai
• Tian Bian
• Guangwu Hu
• Qing Li
• Cheng Huang

Rumors spreading on social media platforms result in potential damages. A precise rumor detection mechanism can help form a healthy public opinion environment. In recent years, deep learning-based rumor detection methods, especially graph model-based ones, have risen and reached promising performance. However, there are several defects in existing methods, which limit models from efficiently utilizing the propagation structure. In this paper, we propose a novel rumor detection model, which has high accuracy and reaches state-of-the-art performance. First, we design a powerful comprehensive rumor feature extractor that explicitly overcomes the restriction of previous Graph Neural Networks-based models. Then, by introducing Kernel Subtree features, our model acquires the capability to learn crucial local features from important nodes. Comparative experiments performed on two real-world social media platforms demonstrate that our work reaches state-of-the-art performance, which outperforms the best baseline with 1. 6% and 1. 9% in accuracy respectively.

IJCAI Conference 2025 Conference Paper

A Survey on Multi-View Knowledge Graph: Generation, Fusion, Applications and Future Directions

• Zihan Yang
• Xiaohui Tao
• Taotao Cai
• Yifu Tang
• Haoran Xie
• Lin Li
• Jianxin Li
• Qing Li

Knowledge Graphs (KGs) have revolutionized structured knowledge representation, yet their capacity to model real-world complexity and heterogeneity remains fundamentally constrained. The emerging paradigm of Multi-View Knowledge Graphs (MVKGs) addresses this gap through multi-view learning, but existing research lacks systematic integration. This survey provides the first systematic consolidation of MVKG methodologies, with four pivotal contributions: 1) The first unified taxonomy of view generation paradigms that rigorously categorizes view into four types: structure, semantic, representation, and knowledge & modality; 2) A novel methodological typology for view fusion that systematically classifies techniques by fusion targets (feature, decision, and hybrid); 3) Task-centric application mapping that bridges theoretical MVKG constructs to node/link/graph-level downstream tasks; 4) A forward-looking roadmap identifying underexplored challenges. By unifying fragmented methodologies and formalizing MVKG design principles, this survey serves as a roadmap for advancing KG versatility in complex AI-driven scenarios. In doing so, it paves the way for more efficient knowledge integration, enhanced decision-making, and cross-domain learning in real-world applications.

EAAI Journal 2025 Journal Article

An effective hybrid algorithm with log-sum regularization and modified momentum for restricted Boltzmann machine

• Huihui Shen
• Hongwei Li
• Zhiguo Gong
• Qing Li

Restricted Boltzmann Machine (RBM) is a strong representation and generative model for unsupervised feature extraction in deep learning. Deep architectures can effectively enhance the capability for image feature expression in image recognition. However, deep architectures have two main disadvantages: 1) they are prone to overfitting, and 2) they are computationally expensive. We propose an effective hybrid algorithm with log-sum regularization and modified momentum for RBM, aiming to overcome overfitting and enhance computational efficiency. Firstly, we improve the gradient approximation by incorporating bias modification. Secondly, we incorporate the log-sum norm into RBM to achieve sparse representation, allowing the sparsity of each hidden unit to be learned adaptively. Thirdly, different combinations of the momentum term are used in the pre-training and fine-tuning phases of the RBM to accelerate the network convergence. Recognition experiments on the MNIST, CMU-PIE, and Extended Yale B databases demonstrate the advantages of our method over other deep learning algorithms in terms of recognition performance and computational efficiency. Denoising experiments for removing the random noise indicate good image denoising capabilities of our proposed method over other unsupervised learning algorithms in natural image data, simulated seismic data, and real seismic data. Based on these image recognition and image denoising results, this novel combination of the improved gradient approximation with modified momentum and the log-sum regularization for training RBM is an effective method for learning robust generative and representation models. Additionally, our research provides a reference for using RBMs to develop engineering applications of artificial intelligence.

EAAI Journal 2025 Journal Article

An image segmentation method for solid-liquid separation on shale shaker based on an improved U2Net

• Wenbin Wang
• Yongjun Hou
• Rui Jiang
• Pan Fang
• Hong Peng
• Qing Li
• Huachuan Li

In the actual production process of shale shakers, detecting the solid-liquid separation state of the screen surface faces numerous challenges, such as difficulty in recognizing the mud boundary, insufficient anti-interference ability, and misjudgment caused by background interference. To address these issues, this paper proposes a screen surface mud image segmentation method based on U2Net, namely CBAM-U2Net. By introducing the Convolutional Block Attention Module (CBAM) and combining it with Multi-layer Recursive Residual Blocks (RSU), a network structure is designed that can efficiently fuse global and local features, significantly improving segmentation accuracy and robustness. The network includes encoder and decoder parts, employing convolution, batch normalization, ReLU activation, and multi-scale feature fusion strategies. Experimental results show that the CBAM-U2Net method demonstrates excellent segmentation performance under various working conditions, achieving outstanding results with mIoU, F1-score, Precision, and Recall at 83. 38%, 89. 75%, 89. 38%, and 92. 64%, respectively, with significantly enhanced anti-interference capability. The CBAM-U2Net method provides an efficient and reliable solution for the intelligent monitoring of the solid-liquid separation state in shale shakers, offering significant practical application value.

NeurIPS Conference 2025 Conference Paper

Cognitive Predictive Processing: A Human-inspired Framework for Adaptive Exploration in Open-World Reinforcement Learning

• boheng liu
• Ziyu Li
• Chenghua Duan
• Yutian Liu
• Zhuo Wang
• Xiuxing Li
• Qing Li
• Xia Wu

Open-world reinforcement learning challenges agents to develop intelligent behavior in vast exploration spaces. Recent approaches like LS-Imagine have advanced the field by extending imagination horizons through jumpy state transitions, yet remain limited by fixed exploration mechanisms and static jump thresholds that cannot adapt across changing task phases, resulting in inefficient exploration and lower completion rates. Humans demonstrate remarkable capabilities in open-world decision-making through a chain-like process of task decomposition, selective memory utilization, and adaptive uncertainty regulation. Inspired by human decision-making processes, we present Cognitive Predictive Processing (CPP), a novel framework that integrates three neurologically-inspired systems: a phase-adaptive cognitive controller that dynamically decomposes tasks into exploration, approach, and completion phases with adaptive parameters; a dual-memory integration system implementing dual-modal memory that balances immediate context with selective long-term storage; and an uncertainty-modulated prediction regulator that continuously updates environmental predictions to modulate exploration behavior. Comprehensive experiments in MineDojo demonstrate that these human-inspired decision-making strategies enhance performance over recent techniques, with success rates improving by an average of 4. 6\% across resource collection tasks while reducing task completion steps by an average of 7. 1\%. Our approach bridges cognitive neuroscience and reinforcement learning, excelling in complex scenarios that require sustained exploration and strategic adaptation while demonstrating how neural-inspired models can solve key challenges in open-world AI systems.

IJCAI Conference 2025 Conference Paper

Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extraction

• Li Yuan
• Yi Cai
• Xudong Shen
• Qing Li
• Qingbao Huang
• Zikun Deng
• Tao Wang

Multimodal Information Extraction (MIE) has gained attention for extracting structured information from multimedia sources. Traditional methods tackle MIE tasks separately, missing opportunities to share knowledge across tasks. Recent approaches unify these tasks into a generation problem using instruction-based T5 models with visual adaptors, optimized through full-parameter fine-tuning. However, this method is computationally intensive, and multi-task fine-tuning often faces gradient conflicts, limiting performance. To address these challenges, we propose collaborative multi-LoRA experts with achievement-based multi-task loss (C-LoRAE) for MIE tasks. C-LoRAE extends the low-rank adaptation (LoRA) method by incorporating a universal expert to learn shared multimodal knowledge from cross-MIE tasks and task-specific experts to learn specialized instructional task features. This configuration enhances the model’s generalization ability across multiple tasks while maintaining the independence of various instruction tasks and mitigating gradient conflicts. Additionally, we propose an achievement-based multi-task loss to balance training progress across tasks, addressing the imbalance caused by varying numbers of training samples in MIE tasks. Experimental results on seven benchmark datasets across three key MIE tasks demonstrate that C-LoRAE achieves superior overall performance compared to traditional fine-tuning methods and LoRA methods while utilizing a comparable number of training parameters to LoRA.

AAAI Conference 2025 Conference Paper

CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization

• Feize Wu
• Yun Pang
• Junyi Zhang
• Lianyu Pang
• Jian Yin
• Baoquan Zhao
• Qing Li
• Xudong Mao

Recent advances in text-to-image personalization have enabled high-quality and controllable image synthesis for user-provided concepts. However, existing methods still struggle to balance identity preservation with text alignment. Our approach is based on the fact that generating prompt-aligned images requires a precise semantic understanding of the prompt, which involves accurately processing the interactions between the new concept and its surrounding context tokens within the CLIP text encoder. To address this, we aim to embed the new concept properly into the input embedding space of the text encoder, allowing for seamless integration with existing tokens. We introduce Context Regularization (CoRe), which enhances the learning of the new concept's text embedding by regularizing its context tokens in the prompt. This is based on the insight that appropriate output vectors of the text encoder for the context tokens can only be achieved if the new concept's text embedding is correctly learned. CoRe can be applied to arbitrary prompts without requiring the generation of corresponding images, thus improving the generalization of the learned text embedding. Additionally, CoRe can serve as a test-time optimization technique to further enhance the generations for specific prompts. Comprehensive experiments demonstrate that our method outperforms several baseline methods in both identity preservation and text alignment.

ECAI Conference 2025 Conference Paper

Don't Stop Pre-Training Small Language Models for Continual Enhancement of Reasoning

• Qing Li
• Qibin Zheng
• Yi Liu 0043
• Xingchun Diao

We investigate the continual enhancement of mathematical reasoning abilities in small language models (SLMs). While large language models (LLMs) demonstrate impressive reasoning performance, their deployment is often constrained by substantial computational costs. Existing approaches to improving SLMs mainly rely on knowledge distillation from costly teacher LLMs, which typically improves mathematical reasoning at the expense of general capabilities. In this work, we show that continual pre-training (CPT) has strong potential to enhance the mathematical reasoning ability of SLMs without relying on large teacher models. We also find that its effectiveness critically depends on the quality of the training data. To maximize efficiency and performance, we propose Dual-Metric Selection for Continual Pre-training (DRIFT), a novel data selection strategy that identifies optimal training data through task-aligned loss differences and distributional regularization. To further enhance task-specific reasoning while preserving general capabilities, we introduce a metadata-aware data mixture that integrates diverse sources during CPT. Extensive experiments on multiple arithmetic reasoning benchmarks demonstrate the effectiveness of DRIFT: SLMs trained with DRIFT achieve substantial gains in reasoning performance, surpassing larger models on specific tasks, while largely preserving general capabilities.

AAAI Conference 2025 Conference Paper

Efficient Robustness Evaluation via Constraint Relaxation

• Chao Pan
• Yu Wu
• Ke Tang
• Qing Li
• Xin Yao

The study of enhancing model robustness against adversarial examples has become increasingly critical in the security of deep learning, leading to the development of numerous adversarial defense techniques. While these defense methods have shown promise in mitigating the impact of adversarial perturbations, evaluating their effectiveness remains a critical challenge. The recently introduced AutoAttack technique has been recognized as a standardized method for assessing model robustness. However, the computational demands of the AutoAttack method significantly limits its applicability, underscoring the urgent need for efficient evaluation techniques. To address this challenge, we propose a novel and efficient evaluation framework based on strategic constraint relaxation. Our key insight is that temporarily expanding the adversarial perturbation bounds during the attack process can help discover more effective adversarial examples. Based on this insight, we develop the Constraint Relaxation Attack (CR Attack) method, which systematically relaxes and resets perturbation constraints during optimization. Extensive experiments on 105 robust models show that CR Attack outperforms AutoAttack in both attack success rate and efficiency, reducing forward and backward propagation time by 38.3× and 15.9× respectively. Through comprehensive analysis, we validate that the constraint relaxation mechanism is crucial for the method's effectiveness.

AAAI Conference 2025 Conference Paper

EvdCLIP: Improving Vision-Language Retrieval with Entity Visual Descriptions from Large Language Models

• GuangHao Meng
• Sunan He
• Jinpeng Wang
• Tao Dai
• Letian Zhang
• Jieming Zhu
• Qing Li
• Gang Wang

Vision-language retrieval (VLR) has attracted significant attention in both academia and industry, which involves using text (or images) as queries to retrieve corresponding images (or text). However, existing methods often neglect the rich visual semantics knowledge of entities, thus leading to incorrect retrieval results. To address this problem, we propose the Entity Visual Description enhanced CLIP (EvdCLIP), designed to leverage the visual knowledge of entities to enrich queries. Specifically, since humans recognize entities through visual cues, we employ a large language model (LLM) to generate Entity Visual Descriptions (EVDs) as alignment cues to complement textual data. These EVDs are then integrated into raw queries to create visually-rich, EVD-enhanced queries. Furthermore, recognizing that EVD-enhanced queries may introduce noise or low-quality expansions, we develop a novel, trainable EVD-aware Rewriter (EaRW) for vision-language retrieval tasks. EaRW utilizes EVD knowledge and the generative capabilities of the language model to effectively rewrite queries. With our specialized training strategy, EaRW can generate high-quality and low-noise EVD-enhanced queries. Extensive quantitative and qualitative experiments on image-text retrieval benchmarks validate the superiority of EvdCLIP on vision-language retrieval tasks.

AAAI Conference 2025 Conference Paper

Explicitly Guided Difficulty-Controllable Visual Question Generation

• Jiayuan Xie
• Mengqiu Cheng
• Xinting Zhang
• Yi Cai
• Guimin Hu
• Mengying Xie
• Qing Li

Visual question generation (VQG) aims to generate questions from images automatically. While existing studies primarily focus on the quality of generated questions, such as fluency and relevance, the difficulty of the questions is also a crucial factor in assessing their quality. Question difficulty directly impacts the effectiveness of VQG systems in applications like education and human-computer interaction, where appropriately challenging questions can stimulate learning interest and improve interaction experiences. However, accurately defining and controlling question difficulty is a challenging task due to its multidimensional and subjective nature. In this paper, we propose a new definition of the difficulty of questions, i.e., being positively correlated with the number of reasoning steps required to answer a question. For our definition, we construct a corresponding dataset and propose a benchmark as a foundation for future research. Our benchmark is designed to progressively increase the reasoning steps involved in generating questions. Specifically, we first extract the relationships among objects in the image to form a reasoning chain, then gradually increase the difficulty by rewriting the generated question to include more reasoning sub-chains. Experimental results on our constructed dataset show that our benchmark significantly outperforms existing baselines in controlling the reasoning chains of generated questions, producing questions with varying difficulty levels.

AAAI Conference 2025 Conference Paper

FIRM: Flexible Interactive Reflection ReMoval

• Xiao Chen
• Xudong Jiang
• Yunkang Tao
• Zhen Lei
• Qing Li
• Chenyang Lei
• Zhaoxiang Zhang

Removing reflection from a single image is challenging due to the absence of general reflection priors. Although existing methods incorporate extensive user guidance for satisfactory performance, they often lack the flexibility to adapt user guidance in different modalities, and dense user interactions further limit their practicality. To alleviate these problems, this paper presents FIRM, a novel framework for Flexible Interactive image Reflection reMoval with various forms of guidance, where users can provide sparse visual guidance (e.g., points, boxes, or strokes) or text descriptions for better reflection removal. Firstly, we design a novel user guidance conversion module (UGC) to transform different forms of guidance into unified contrastive masks. The contrastive masks provide explicit cues for identifying reflection and transmission layers in blended images. Secondly, we devise a contrastive mask-guided reflection removal network that comprises a newly proposed contrastive guidance interaction block (CGIB). This block leverages a unique cross-attention mechanism that merges contrastive masks with image features, allowing for precise layer separation. The proposed framework requires only 10% of the guidance time needed by previous interactive methods, which makes a step-change in flexibility. Extensive results on public real-world reflection removal datasets validate that our method demonstrates state-of-the-art reflection removal performance.

NeurIPS Conference 2025 Conference Paper

From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes

• Tianxu Wang
• Zhuofan Zhang
• Ziyu Zhu
• Yue Fan
• Jing Xiong
• Pengxiang Li
• Xiaojian (Shawn) Ma
• Qing Li

3D visual grounding has made notable progress in localizing objects within complex 3D scenes. However, grounding referring expressions beyond objects in 3D scenes remains unexplored. In this paper, we introduce Anywhere3D-Bench, a holistic 3D visual grounding benchmark consisting of 2, 886 referring expression-3D bounding box pairs spanning four different grounding levels: human-activity areas, unoccupied space beyond objects, individual objects in the scene, and fine-grained object parts. We assess a range of state-of-the-art 3D visual grounding methods alongside large language models (LLMs) and multimodal LLMs (MLLMs) on Anywhere3D-Bench. Experimental results reveal that space-level and part-level visual grounding pose the greatest challenges: space-level tasks require a more comprehensive spatial reasoning ability, for example, modeling distances and spatial relations within 3D space, while part-level tasks demand fine-grained perception of object composition. Even the best performance model, OpenAI o4-mini, achieves only 23. 00% accuracy on space-level tasks and 31. 46% on part-level tasks, significantly lower than its performance on area-level and object-level tasks. These findings underscore a critical gap in current models’ capacity to understand and reason about 3D scenes beyond object-level semantics.

TIST Journal 2025 Journal Article

Graph Machine Learning in the Era of Large Language Models (LLMs)

• Shijie Wang
• Jiani Huang
• Zhikai Chen
• Yu Song
• Wenzhuo Tang
• Haitao Mao
• Wenqi Fan
• Hui Liu

Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graphs. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications, such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML’s generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations, such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph Heterophily and Out-of-Distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.

AAAI Conference 2025 Conference Paper

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

• Qing Li
• Jiahui Geng
• Derui Zhu
• Zongxiong Chen
• Kun Song
• Lei Ma
• Fakhri Karray

Warning: This paper contains offensive content that may disturb some readers. Vision-language models (VLMs) demonstrate strong multimodal capabilities but have been found to be more susceptible to generating harmful content compared to their backbone large language models (LLMs). Our investigation reveals that the integration of images significantly shifts the model's internal activations during the forward pass, diverging from those triggered by textual input. Moreover, the safety alignments of LLMs embedded within VLMs are not sufficiently robust to handle the activations discrepancies, making the models vulnerable to even the simplest jailbreaking attacks. To address this issue, we propose an internal activation revision approach that efficiently revises activations during generation, steering the model toward safer outputs. Our framework incorporates revisions at both the layer and head levels, offering control over the model's generation at varying levels of granularity. In addition, we explore three strategies for constructing positive and negative samples and two approaches for extracting revision vectors, resulting in different variants of our method. Comprehensive experiments demonstrate that the internal activation revision method significantly improves the safety of widely used VLMs, reducing attack success rates by an average of 48.94%, 34.34%, 43.92%, and 52.98% on SafeBench, Safe-Unsafe, Unsafe, and MM-SafetyBench, respectively, while minimally impacting model helpfulness.

NeurIPS Conference 2025 Conference Paper

Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning

• Pengxiang Li
• Zhi Gao
• Bofei Zhang
• Yapeng Mi
• Xiaojian (Shawn) Ma
• Chenrui Shi
• Tao Yuan
• Yuwei Wu

Multimodal agents, which integrate a controller (e. g. , a vision language model) with external tools, have demonstrated remarkable capabilities in tackling complex multimodal tasks. Existing approaches for training these agents, both supervised fine-tuning and reinforcement learning, depend on extensive human-annotated task-answer pairs and tool trajectories. However, for complex multimodal tasks, such annotations are prohibitively expensive or impractical to obtain. In this paper, we propose an iterative tool usage exploration method for multimodal agents without any pre-collected data, namely SPORT, via step-wise preference optimization to refine the trajectories of tool usage. Our method enables multimodal agents to autonomously discover effective tool usage strategies through self-exploration and optimization, eliminating the bottleneck of human annotation. SPORT has four iterative components: task synthesis, step sampling, step verification, and preference tuning. We first synthesize multimodal tasks using language models. Then, we introduce a novel trajectory exploration scheme, where step sampling and step verification are executed alternately to solve synthesized tasks. In step sampling, the agent tries different tools and obtains corresponding results. In step verification, we employ a verifier to provide AI feedback to construct step-wise preference data. The data is subsequently used to update the controller for tool usage through preference tuning, producing a SPORT agent. By interacting with real environments, the SPORT agent gradually evolves into a more refined and capable system. Evaluation in the GTA and GAIA benchmarks shows that the SPORT agent achieves 6. 41% and 3. 64% improvements, underscoring the generalization and effectiveness introduced by our method.

NeurIPS Conference 2025 Conference Paper

Learning to Plan Like the Human Brain via Visuospatial Perception and Semantic-Episodic Synergistic Decision-Making

• Tianyuan Jia
• Ziyu Li
• Qing Li
• Xiuxing Li
• Xiang Li
• Chen Wei
• Li Yao
• Xia Wu

Motion planning in high-dimensional continuous spaces remains challenging due to complex environments and computational constraints. Although learning-based planners, especially graph neural network (GNN)-based, have significantly improved planning performance, they still struggle with inaccurate graph construction and limited structural reasoning, constraining search efficiency and path quality. The human brain exhibits efficient planning through a two-stage Perception-Decision model. First, egocentric spatial representations from visual and proprioceptive input are constructed, and then semantic–episodic synergy is leveraged to support decision-making in uncertainty scenarios. Inspired by this process, we propose NeuroMP, a brain-inspired planning framework that learns to plan like the human brain. NeuroMP integrates a Perceptive Segment Selector inspired by visuospatial perception to construct safer graphs, and a Global Alignment Heuristic guide search in weakly connected graphs by modeling semantic-episodic synergistic decision-making. Experimental results demonstrate that NeuroMP significantly outperforms existing planning methods in efficiency and quality while maintaining a high success rate.

NeurIPS Conference 2025 Conference Paper

NEP: Autoregressive Image Editing via Next Editing Token Prediction

• Huimin Wu
• Xiaojian (Shawn) Ma
• Haozhe Zhao
• Yanpeng Zhao
• Qing Li

Text-guided image editing involves modifying a source image based on a language instruction and, typically, requires changes to only small local regions. However, existing approaches generate the entire target image rather than selectively regenerate only the intended editing areas. This results in (1) unnecessary computational costs and (2) a bias toward reconstructing non-editing regions, which compromises the quality of the intended edits. To resolve these limitations, we propose to formulate image editing as $\textbf{N}$ext $\textbf{E}$diting-token $\textbf{P}$rediction (NEP) based on autoregressive image generation, where only regions that need to be edited are regenerated, thus avoiding unintended modification to the non-editing areas. To enable any-region editing, we propose to pre-train an any-order autoregressive text-to-image (T2I) model. Once trained, it is capable of zero-shot image editing and can be easily adapted to NEP for image editing, which achieves a new state-of-the-art on widely used image editing benchmarks. Moreover, our model naturally supports test-time scaling (TTS) through iteratively refining its generation in a zero-shot manner.

YNIMG Journal 2025 Journal Article

ODF based deep learning network for unsupervised deformable diffusion resonance image registration (ODDRnet)

• Mengyao Li
• Jieying Zhang
• Baogui Zhang
• Kai Liu
• Qing Li
• Peipeng Liang
• Wenzhang Liu
• Tianyi Qian

In recent years, deep-learning algorithms have been widely adopted for medical-image registration owing to their high speed and accuracy, yet most works focus on scalar images. Diffusion MRI (dMRI) is primarily used to depict white-matter architecture in the brain. Because each voxel contains measurements acquired under multiple diffusion directions and b-values, dMRI constitutes a high-dimensional imaging modality. Existing techniques often neglect the directional information, particularly in regions with complex fiber crossings, compromising the alignment of white matter structures between individuals and potentially affecting subsequent analyses like tractography. Aligning Fiber Orientation Distribution Functions (fODF), which represent the distribution of fiber orientations per voxel, is a promising strategy to leverage directional information. This approach facilitates more anatomically precise correspondence of white matter pathways compared to methods relying solely on scalar information. This study proposes ODDRnet, an end-to-end, unsupervised deep-learning framework for nonlinear registration of dMRI data. ODDRnet directly aligns high-dimensional fODF derived from raw dMRI signals. It predicts a dense deformation field which is used to spatially warp the fODFs while appropriately reorienting the directional information, ensuring accurate alignment. ODDRnet was evaluated on multiple public and private datasets and compared with several state-of-the-art methods. Experimental results demonstrated that ODDRnet achieved superior macroscopic accuracy, with a mean increase of 0.02 in the tract dice and a reduction of 0.31 mm in tract distance. Furthermore, ODDRnet showed robust generalization across diverse ethnicities, age groups, health conditions, and imaging protocols, confirming its adaptability for various dMRI registration tasks.

NeurIPS Conference 2025 Conference Paper

PairEdit: Learning Semantic Variations for Exemplar-based Image Editing

• Haoguang Lu
• Jiacheng Chen
• Zhenguo Yang
• Aurele Gnanha
• Fu Lee Wang
• Qing Li
• Xudong Mao

Recent advancements in text-guided image editing have achieved notable success by leveraging natural language prompts for fine-grained semantic control. However, certain editing semantics are challenging to specify precisely using textual descriptions alone. A practical alternative involves learning editing semantics from paired source-target examples. Existing exemplar-based editing methods still rely on text prompts describing the change within paired examples or learning implicit text-based editing instructions. In this paper, we introduce PairEdit, a novel visual editing method designed to effectively learn complex editing semantics from a limited number of image pairs or even a single image pair, without using any textual guidance. We propose a target noise prediction that explicitly models semantic variations within paired images through a guidance direction term. Moreover, we introduce a content-preserving noise schedule to facilitate more effective semantic learning. We also propose optimizing distinct LoRAs to disentangle the learning of semantic variations from content. Extensive qualitative and quantitative evaluations demonstrate that PairEdit successfully learns intricate semantics while significantly improving content consistency compared to baseline methods. Code is available at https: //github. com/xudonmao/PairEdit.

AAAI Conference 2025 Conference Paper

Revolutionizing Encrypted Traffic Classification with MH-Net: A Multi-View Heterogeneous Graph Model

• Haozhen Zhang
• Haodong Yue
• Xi Xiao
• Le Yu
• Qing Li
• Zhen Ling
• Ye Zhang

With the growing significance of network security, the classification of encrypted traffic has emerged as an urgent challenge. Traditional byte-based traffic analysis methods are constrained by the rigid granularity of information and fail to fully exploit the diverse correlations between bytes. To address these limitations, this paper introduces MH-Net, a novel approach for classifying network traffic that leverages multi-view heterogeneous traffic graphs to model the intricate relationships between traffic bytes. The essence of MH-Net lies in aggregating varying numbers of traffic bits into multiple types of traffic units, thereby constructing multi-view traffic graphs with diverse information granularities. By accounting for different types of byte correlations, such as header-payload relationships, MH-Net further endows the traffic graph with heterogeneity, significantly enhancing model performance. Notably, we employ contrastive learning in a multi-task manner to strengthen the robustness of the learned traffic unit representations. Experiments conducted on the ISCX and CIC-IoT datasets for both the packet-level and flow-level traffic classification tasks demonstrate that MH-Net achieves the best overall performance compared to dozens of SOTA methods.

NeurIPS Conference 2025 Conference Paper

SteerConf: Steering LLMs for Confidence Elicitation

• Ziang Zhou
• Tianyuan Jin
• Jieming Shi
• Qing Li

Large Language Models (LLMs) exhibit impressive performance across diverse domains but often suffer from overconfidence, limiting their reliability in critical applications. We propose SteerConf, a novel framework that systematically steers LLMs' confidence scores to improve their calibration and reliability. SteerConf introduces three key components: (1) a steering prompt strategy that guides LLMs to produce confidence scores in specified directions (e. g. , conservative or optimistic) by leveraging prompts with varying steering levels; (2) a steered confidence consistency measure that quantifies alignment across multiple steered confidences to enhance calibration; and (3) a steered confidence calibration method that aggregates confidence scores using consistency measures and applies linear quantization for answer selection. SteerConf operates without additional training or fine-tuning, making it broadly applicable to existing LLMs. Experiments on seven benchmarks spanning professional knowledge, common sense, ethics, and reasoning tasks, using advanced LLM models (GPT-3. 5, LLaMA 3, GPT-4), demonstrate that SteerConf significantly outperforms existing methods, often by a significant margin. Our findings highlight the potential of steering the confidence of LLMs to enhance their reliability for safer deployment in real-world applications. The implementation is at \url{https: //github. com/scottjiao/SteerConf}.

TMLR Journal 2025 Journal Article

The AI Hippocampus: How Far are We From Human Memory?

• Zixia Jia
• Jiaqi Li
• Yipeng Kang
• Yuxuan Wang
• Tong Wu
• Quansen Wang
• Xiaobo Wang
• Shuyi Zhang

Memory plays a foundational role in augmenting the reasoning, adaptability, and contextual fidelity of modern Large Language Models (LLMs) and Multi-Modal LLMs (MLLMs). As these models transition from static predictors to interactive systems capable of continual learning and personalized inference, the incorporation of memory mechanisms has emerged as a central theme in their architectural and functional evolution. This survey presents a comprehensive and structured synthesis of memory in LLMs and MLLMs, organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms. Specifically, the survey delineates three primary memory frameworks. \textit{Implicit memory} refers to the knowledge embedded within the internal parameters of pre-trained transformers, encompassing their capacity for memorization, associative retrieval, and contextual reasoning. Recent work has explored methods to interpret, manipulate, and reconfigure this latent memory. \textit{Explicit memory} involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations—such as textual corpora, dense vectors, and graph-based structures—thereby enabling scalable and updatable interaction with information sources. \textit{Agentic memory} introduces persistent, temporally extended memory structures within autonomous agents, facilitating long-term planning, self-consistency, and collaborative behavior in multi-agent systems, with relevance to embodied and interactive AI. Extending beyond text, the survey examines the integration of memory within multi-modal settings, where coherence across vision, language, audio, and action modalities is essential. Key architectural advances, benchmark tasks, and open challenges are discussed, including issues related to memory capacity, alignment, factual consistency, and cross-system interoperability. By charting the current landscape and identifying critical research directions, this survey aims to inform the development of memory-augmented (M)LLMs that are more flexible, context-sensitive, and aligned with the requirements of real-world intelligent systems. The survey’s website is available at \url{https://github.com/bigai-nlco/LLM-Memory-Survey}.

JBHI Journal 2025 Journal Article

Towards Clinically Applicable Large-Model-Based Privacy-Preserving Polyp Segmentation: A Federated LoRA Approach to Colonoscopy

• Xingchi Chen
• Fa Zhu
• Dazhou Li
• Qing Li
• Muhammad Shahid Anwar
• Gaoyang Shan
• Jun Jiang

Colonoscopy polyp segmentation is essential for accurate lesion detection and workflow optimization in clinical practice. However, deploying large foundation models in medical settings faces challenges related to patient privacy, computational overhead, and heterogeneous data distributions. In this study, we propose PolypSAMFL, a novel framework that integrates lowrank adaptation (LoRA) into the Segment Anything Model (SAM) within a federated learning paradigm to deliver privacypreserving, highprecision polyp segmentation. By freezing the majority of SAM's pretrained parameters and finetuning only compact LoRA modules in the image encoder and mask decoder, PolypSAMFL significantly reduces communication costs while maintaining robust feature extraction across distributed clinical datasets. We further propose a boundaryaware loss function and a multiresolution mask synthesis strategy to enhance delineation of irregular and lowcontrast polyp boundaries. Extensive evaluation on four public colonoscopy datasets demonstrates that our method yields a mean Dice score of 0. 987 and intersectionoverunion of 0. 976, outperforming stateoftheart approaches while fully preserving data locality. This translates directly to more reliable identification of polyp during colonoscopy. These results validate the clinical utility of PolypSAMFL for realworld, AIdriven colonoscopy workflows, offering a scalable solution that aligns with privacy regulations and resource constraints in modern healthcare environments.

IJCAI Conference 2025 Conference Paper

Tree-of-AdEditor: Heuristic Tree Reasoning for Automated Video Advertisement Editing with Large Language Model

• Yuqi Zhang
• Bin Guo
• Nuo Li
• Ying Zhang
• Shijie Wang
• Zhiwen Yu
• Qing Li

Video advertising has become a popular marketing strategy on e-commerce platforms, requiring high-level semantic reasoning like selling point discovery, narrative organization. Previous rule-based methods struggle with these complex tasks, and learning-based approaches demand large datasets and high training costs. Recently, Large Language Models have opened incredible opportunities for advancing intelligent video advertisement editing. However, Input-output (IO) prompting and Chain-of-Thought (CoT) struggle to adapt to the nonlinear thinking hierarchy of video editing, where editors iteratively select shots or revert them to explore potential editing solutions. While Tree-of-Thought (ToT) offers a conceptual structure that mirrors this hierarchy, it falls short in aligning with effective video advertising strategies and lacks robust fact-checking mechanisms. To address these, we propose a novel framework, Tree-of-AdEditor (ToAE), which constructs a reasoning tree to mimic human editors, and incorporates domain-specific theories and heuristic fact-checking to identify optimal editing solutions. Specifically, motivated by effective advertisement principles, we develop a "local-global" mechanism to guide LLM in both the shot level and sequence level decision-making. We introduce a visual incoherence pruning module to provide external heuristic fact-checking, ensuring visual attractiveness and reducing computation costs. Quantitative experiments and expert evaluation demonstrate the superiority of our method compared to baselines.

NeurIPS Conference 2025 Conference Paper

VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment

• Qing Li
• Huifang Feng
• Xun Gong
• Yu-Shen Liu

3D Gaussian Splatting has recently emerged as an efficient solution for high-quality and real-time novel view synthesis. However, its capability for accurate surface reconstruction remains underexplored. Due to the discrete and unstructured nature of Gaussians, supervision based solely on image rendering loss often leads to inaccurate geometry and inconsistent multi-view alignment. In this work, we propose a novel method that enhances the geometric representation of 3D Gaussians through view alignment (VA). Specifically, we incorporate edge-aware image cues into the rendering loss to improve surface boundary delineation. To enforce geometric consistency across views, we introduce a visibility-aware photometric alignment loss that models occlusions and encourages accurate spatial relationships among Gaussians. To further mitigate ambiguities caused by lighting variations, we incorporate normal-based constraints to refine the spatial orientation of Gaussians and improve local surface estimation. Additionally, we leverage deep image feature embeddings to enforce cross-view consistency, enhancing the robustness of the learned geometry under varying viewpoints and illumination. Extensive experiments on standard benchmarks demonstrate that our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis. The source code is available at https: //github. com/LeoQLi/VA-GS.

EAAI Journal 2024 Journal Article

Accurate identification and confidence evaluation of automatic generation control command execution effect based on deep learning fusion model

• Guangyu Chen
• Hongtong Liu
• Haiyang Jiang
• Qing Li
• Yangfei Zhang
• Sipeng Hao
• Wenhe Zhao

With the increasing complexity of the power grid, the precision of the thermal power units' execution of automatic generation control (AGC) commands is gradually increasing the impact on the online regulation of the power grid. The deviation between the actual output of thermal power units and the AGC command of the grid will not only affect the consumption of new energy output, but also endanger the safe operation of the grid. This paper introduces “deep learning” technology to solve the problem. Firstly, an AGC command execution effect identification and confidence evaluation algorithm (ACEEI-CEA) is proposed. The algorithm builds a neural network model to accurately predict the unit output and the confidence evaluation of the prediction results. Next, a high-dimensional input preprocessing strategy based on variational autoencoder (VAE) is proposed to reduce the dimensionality of the model input attributes, improving the convergence and accuracy of the model. Finally, an AGC optimal command fast inversion solution method (AOCFISM) is designed. This method transforms the unit output deviation problem into an objective optimisation problem. And improve the efficiency of solving the optimal AGC command value by constraining the unit command value. The calculation results show that the error of the prediction results of the model proposed in this paper is 5% lower than that of the traditional neural network. The difference between the output value of the optimal AGC command and the expected output value obtained is less than 0. 5 MV, which can support AGC online decision-making. © 2017 Elsevier Inc. All rights reserved.

AAAI Conference 2024 Conference Paper

Adversarial Initialization with Universal Adversarial Perturbation: A New Approach to Fast Adversarial Training

• Chao Pan
• Qing Li
• Xin Yao

Traditional adversarial training, while effective at improving machine learning model robustness, is computationally intensive. Fast Adversarial Training (FAT) addresses this by using a single-step attack to generate adversarial examples more efficiently. Nonetheless, FAT is susceptible to a phenomenon known as catastrophic overfitting, wherein the model's adversarial robustness abruptly collapses to zero during the training phase. To address this challenge, recent studies have suggested adopting adversarial initialization with Fast Gradient Sign Method Adversarial Training (FGSM-AT), which recycles adversarial perturbations from prior epochs by computing gradient momentum. However, our research has uncovered a flaw in this approach. Given that data augmentation is employed during the training phase, the samples in each epoch are not identical. Consequently, the method essentially yields not the adversarial perturbation of a singular sample, but rather the Universal Adversarial Perturbation (UAP) of a sample and its data augmentation. This insight has led us to explore the potential of using UAPs for adversarial initialization within the context of FGSM-AT. We have devised various strategies for adversarial initialization utilizing UAPs, including single, class-based, and feature-based UAPs. Experiments conducted on three distinct datasets demonstrate that our method achieves an improved trade-off among robustness, computational cost, and memory footprint. Code is available at https://github.com/fzjcdt/fgsm-uap.

NeurIPS Conference 2024 Conference Paper

AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation

• Lianyu Pang
• Jian Yin
• Baoquan Zhao
• Feize Wu
• Fu Lee Wang
• Qing Li
• Xudong Mao

Recent advances in text-to-image models have enabled high-quality personalized image synthesis based on user-provided concepts with flexible textual control. In this work, we analyze the limitations of two primary techniques in text-to-image personalization: Textual Inversion and DreamBooth. When integrating the learned concept into new prompts, Textual Inversion tends to overfit the concept, while DreamBooth often overlooks it. We attribute these issues to the incorrect learning of the embedding alignment for the concept. To address this, we introduce AttnDreamBooth, a novel approach that separately learns the embedding alignment, the attention map, and the subject identity across different training stages. We also introduce a cross-attention map regularization term to enhance the learning of the attention map. Our method demonstrates significant improvements in identity preservation and text alignment compared to the baseline methods.

AAAI Conference 2024 Conference Paper

Automated Defect Report Generation for Enhanced Industrial Quality Control

• Jiayuan Xie
• Zhiping Zhou
• Zihan Wu
• Xinting Zhang
• Jiexin Wang
• Yi Cai
• Qing Li

Defect detection is a pivotal aspect ensuring product quality and production efficiency in industrial manufacturing. Existing studies on defect detection predominantly focus on locating defects through bounding boxes and classifying defect types. However, their methods can only provide limited information and fail to meet the requirements for further processing after detecting defects. To this end, we propose a novel task called defect detection report generation, which aims to provide more comprehensive and informative insights into detected defects in the form of text reports. For this task, we propose some new datasets, which contain 16 different materials and each defect contains a detailed report of human constructs. In addition, we propose a knowledge-aware report generation model as a baseline for future research, which aims to incorporate additional knowledge to generate detailed analysis and subsequent processing related to defect in images. By constructing defect report datasets and proposing corresponding baselines, we chart new directions for future research and practical applications of this task.

AAAI Conference 2024 Conference Paper

Compositional Inversion for Stable Diffusion Models

• Xulu Zhang
• Xiao-Yong Wei
• Jinlin Wu
• Tianyi Zhang
• Zhaoxiang Zhang
• Zhen Lei
• Qing Li

Inversion methods, such as Textual Inversion, generate personalized images by incorporating concepts of interest provided by user images. However, existing methods often suffer from overfitting issues, where the dominant presence of inverted concepts leads to the absence of other desired concepts. It stems from the fact that during inversion, the irrelevant semantics in the user images are also encoded, forcing the inverted concepts to occupy locations far from the core distribution in the embedding space. To address this issue, we propose a method that guides the inversion process towards the core distribution for compositional embeddings. Additionally, we introduce a spatial regularization approach to balance the attention on the concepts being composed. Our method is designed as a post-training approach and can be seamlessly integrated with other inversion methods. Experimental results demonstrate the effectiveness of our proposed approach in mitigating the overfitting problem and generating more diverse and balanced compositions of concepts in the synthesized images. The source code is available at https://github.com/zhangxulu1996/Compositional-Inversion.

YNIMG Journal 2024 Journal Article

Decoding the task specificity of post-error adjustments: Features and determinants

• Qing Li
• Jing Wang
• Zong Meng
• Yongqiang Chen
• Mengke Zhang
• Na Hu
• Xu Chen
• Antao Chen

Errors typically trigger post-error adjustments aimed at improving subsequent reactions within a single task, but little work has focused on whether these adjustments are task-general or task-specific across different tasks. We collected behavioral and electrophysiological (EEG) data when participants performed a psychological refractory period paradigm. This paradigm required them to complete Task 1 and Task 2 separated by a variable stimulus onset asynchrony (SOA). Behaviorally, post-error slowing and post-error accuracy exhibited task-general features at short SOAs but some task-specific features at long SOAs. EEG results manifest that task-general adjustments had a short-lived effect, whereas task-specific adjustments were long-lasting. Moreover, error awareness specifically conduced to the improvement of subsequent sensory processing and behavior performance in Task 1 (the task where errors occurred). These findings demonstrate that post-error adjustments rely on both transient, task-general interference and longer-lasting, task-specific control mechanisms simultaneously, with error awareness playing a crucial role in determining these mechanisms. We further discuss the contribution of central resources to the task specificity of post-error adjustments.

NeurIPS Conference 2024 Conference Paper

Dissect Black Box: Interpreting for Rule-Based Explanations in Unsupervised Anomaly Detection

• Yu Zhang
• Ruoyu Li
• Nengwu Wu
• Qing Li
• Xinhan Lin
• Yang Hu
• Tao Li
• Yong Jiang

In high-stakes sectors such as network security, IoT security, accurately distinguishing between normal and anomalous data is critical due to the significant implications for operational success and safety in decision-making. The complexity is exacerbated by the presence of unlabeled data and the opaque nature of black-box anomaly detection models, which obscure the rationale behind their predictions. In this paper, we present a novel method to interpret the decision-making processes of these models, which are essential for detecting malicious activities without labeled attack data. We put forward the Segmentation Clustering Decision Tree (SCD-Tree), designed to dissect and understand the structure of normal data distributions. The SCD-Tree integrates predictions from the anomaly detection model into its splitting criteria, enhancing the clustering process with the model's insights into anomalies. To further refine these segments, the Gaussian Boundary Delineation (GBD) algorithm is employed to define boundaries within each segmented distribution, effectively delineating normal from anomalous data points. At this point, this approach addresses the curse of dimensionality by segmenting high-dimensional data and ensures resilience to data drift and perturbations through flexible boundary fitting. We transform the intricate operations of anomaly detection into an interpretable rule's format, constructing a comprehensive set of rules for understanding. Our method's evaluation on diverse datasets and models demonstrates superior explanation accuracy, fidelity, and robustness over existing method, proving its efficacy in environments where interpretability is paramount.

NeurIPS Conference 2024 Conference Paper

Entity Alignment with Noisy Annotations from Large Language Models

• Shengyuan Chen
• Qinggang Zhang
• Junnan Dong
• Wen Hua
• Qing Li
• Xiao Huang

Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency.

NeurIPS Conference 2024 Conference Paper

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

• Pengxiang Li
• Zhi Gao
• Bofei Zhang
• Tao Yuan
• Yuwei Wu
• Mehrtash Harandi
• Yunde Jia
• Song-Chun Zhu

Vision language models (VLMs) have achieved impressive progress in diverse applications, becoming a prevalent research direction. In this paper, we build FIRE, a feedback-refinement dataset, consisting of 1. 1M multi-turn conversations that are derived from 27 source datasets, empowering VLMs to spontaneously refine their responses based on user feedback across diverse tasks. To scale up the data collection, FIRE is collected in two components: FIRE-100K and FIRE-1M, where FIRE-100K is generated by GPT-4V, and FIRE-1M is freely generated via models trained on FIRE-100K. Then, we build FIRE-Bench, a benchmark to comprehensively evaluate the feedback-refining capability of VLMs, which contains 11K feedback-refinement conversations as the test data, two evaluation settings, and a model to provide feedback for VLMs. We develop the FIRE-LLaVA model by fine-tuning LLaVA on FIRE-100K and FIRE-1M, which shows remarkable feedback-refining capability on FIRE-Bench and outperforms untrained VLMs by 50%, making more efficient user-agent interactions and underscoring the significance of the FIRE dataset.

JBHI Journal 2024 Journal Article

Multi-Scale Spatio-Temporal Fusion With Adaptive Brain Topology Learning for fMRI Based Neural Decoding

• Ziyu Li
• Qing Li
• Zhiyuan Zhu
• Zhongyi Hu
• Xia Wu

Neural decoding aims to extract information from neurons' activities to reveal how the brain functions. Due to the inherent spatial and temporal characteristics of brain signals, spatio-temporal computing has become a hot topic for neural decoding. However, the extant spatio-temporal decoding methods usually use static brain topology, ignoring the dynamic patterns of the interaction between brain regions. Further, they do not identify the hierarchical organization of brain topology, leading to only superficial insight into brain spatio-temporal interactions. Therefore, here we propose a novel framework, the Multi-Scale Spatio-Temporal framework with Adaptive Brain Topology Learning (MSST-ABTL), for neural decoding. It includes two new capabilities to enhance spatio-temporal decoding: i) ABTL module, which learns dynamic brain topology while updating specific patterns of brain regions, ii) MSST module, which captures the association of spatial pattern and temporal evolution, and further enhances the interpretability of the learned dynamic topology from multi-scale perspective. We evaluated the framework on the public Human Connectome Project (HCP) dataset (resting-state and task-related fMRI data). The extensive experiments show that the proposed MSST-ABTL outperforms state-of-the-art methods on four evaluation metrics, and also can renew the neuroscientific discoveries in the brain's hierarchical patterns.

NeurIPS Conference 2024 Conference Paper

OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents

• Zihao Wang
• Shaofei Cai
• Zhancun Mu
• Haowei Lin
• Ceyao Zhang
• Xuejie Liu
• Qing Li
• Anji Liu

This paper presents OmniJARVIS, a novel Vision-Language-Action (VLA) model for open-world instruction-following agents in Minecraft. Compared to prior works that either emit textual goals to separate controllers or produce the control command directly, OmniJARVIS seeks a different path to ensure both strong reasoning and efficient decision-making capabilities via unified tokenization of multimodal interaction data. First, we introduce a self-supervised approach to learn a behavior encoder that produces discretized tokens for behavior trajectories $\tau = \{o_0, a_0, \dots\}$ and an imitation learning policy decoder conditioned on these tokens. These additional behavior tokens will be augmented to the vocabulary of pretrained Multimodal Language Models. With this encoder, we then pack long-term multimodal interactions involving task instructions, memories, thoughts, observations, textual responses, behavior trajectories, etc into unified token sequences and model them with autoregressive transformers. Thanks to the semantically meaningful behavior tokens, the resulting VLA model, OmniJARVIS, can reason (by producing chain-of-thoughts), plan, answer questions, and act (by producing behavior tokens for the imitation learning policy decoder). OmniJARVIS demonstrates excellent performances on a comprehensive collection of atomic, programmatic, and open-ended tasks in open-world Minecraft. Our analysis further unveils the crucial design principles in interaction data formation, unified tokenization, and its scaling potentials. The dataset, models, and code will be released at https: //craftjarvis. org/OmniJARVIS.

AAAI Conference 2024 Conference Paper

One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training

• Lianbo Ma
• Yuee Zhou
• Jianlun Ma
• Guo Yu
• Qing Li

Weight quantization is an effective technique to compress deep neural networks for their deployment on edge devices with limited resources. Traditional loss-aware quantization methods commonly use the quantized gradient to replace the full-precision gradient. However, we discover that the gradient error will lead to an unexpected zig-zagging-like issue in the gradient descent learning procedures, where the gradient directions rapidly oscillate or zig-zag, and such issue seriously slows down the model convergence. Accordingly, this paper proposes a one-step forward and backtrack way for loss-aware quantization to get more accurate and stable gradient direction to defy this issue. During the gradient descent learning, a one-step forward search is designed to find the trial gradient of the next-step, which is adopted to adjust the gradient of current step towards the direction of fast convergence. After that, we backtrack the current step to update the full-precision and quantized weights through the current-step gradient and the trial gradient. A series of theoretical analysis and experiments on benchmark deep models have demonstrated the effectiveness and competitiveness of the proposed method, and our method especially outperforms others on the convergence performance.

TIST Journal 2024 Journal Article

Optimal Treatment Strategies for Critical Patients with Deep Reinforcement Learning

• Simi Job
• Xiaohui Tao
• Lin Li
• Haoran Xie
• Taotao Cai
• Jianming Yong
• Qing Li

Personalized clinical decision support systems are increasingly being adopted due to the emergence of data-driven technologies, with this approach now gaining recognition in critical care. The task of incorporating diverse patient conditions and treatment procedures into critical care decision-making can be challenging due to the heterogeneous nature of medical data. Advances in Artificial Intelligence (AI), particularly Reinforcement Learning (RL) techniques, enables the development of personalized treatment strategies for severe illnesses by using a learning agent to recommend optimal policies. In this study, we propose a Deep Reinforcement Learning (DRL) model with a tailored reward function and an LSTM-GRU-derived state representation to formulate optimal treatment policies for vasopressor administration in stabilizing patient physiological states in critical care settings. Using an ICU dataset and the Medical Information Mart for Intensive Care (MIMIC-III) dataset, we focus on patients with Acute Respiratory Distress Syndrome (ARDS) that has led to Sepsis, to derive optimal policies that can prioritize patient recovery over patient survival. Both the DDQN ( RepDRL-DDQN ) and Dueling DDQN ( RepDRL-DDDQN ) versions of the DRL model surpass the baseline performance, with the proposed model’s learning agent achieving an optimal learning process across our performance measuring schemes. The robust state representation served as the foundation for enhancing the model’s performance, ultimately providing an optimal treatment policy focused on rapid patient recovery.

JBHI Journal 2024 Journal Article

Timely ICU Outcome Prediction Utilizing Stochastic Signal Analysis and Machine Learning Techniques with Readily Available Vital Sign Data

• Shaodong Wang
• Yiqun Jiang
• Qing Li
• Wenli Zhang

The ICU is a specialized hospital department that offers critical care to patients at high risk. The massive burden of ICU-requiring care requires accurate and timely ICU outcome predictions for alleviating the economic and healthcare burdens imposed by critical care needs. Existing research faces challenges such as feature extraction difficulties, low accuracy, and resource-intensive features. Some studies have explored deep learning models that utilize raw clinical inputs. However, these models are considered non-interpretable black boxes, which prevents their wide application. The objective of the study is to develop a new method using stochastic signal analysis and machine learning techniques to effectively extract features with strong predictive power from ICU patients’ real-time time series of vital signs for accurate and timely ICU outcome prediction. The results show the proposed method extracted meaningful features and outperforms baseline methods, including APACHE IV (AUC = 0. 750), deep learning-based models (AUC = 0. 732, 0. 712, 0. 698, 0. 722), and statistical feature classification methods (AUC = 0. 765) by a large margin (AUC = 0. 869). The proposed method has clinical, management, and administrative implications since it enables healthcare professionals to identify deviations from prognostications timely and accurately and, therefore, to conduct proper interventions.

NeurIPS Conference 2024 Conference Paper

UltraEdit: Instruction-based Fine-Grained Image Editing at Scale

• Haozhe Zhao
• Xiaojian Ma
• Liang Chen
• Shuzheng Si
• Rujie Wu
• Kaikai An
• Peiyu Yu
• Minjia Zhang

This paper presents UltraEdit, a large-scale (~ 4M editing samples), automatically generated dataset for instruction-based image editing. Our key idea is to address the drawbacks in existing image editing datasets like InstructPix2Pix and MagicBrush, and provide a systematic approach to producing massive and high-quality image editing samples: 1) UltraEdit includes more diverse editing instructions by combining LLM creativity and in-context editing examples by human raters; 2) UltraEdit is anchored on real images (photographs or artworks), which offers more diversity and less biases than those purely synthesized by text-to-image models; 3) UltraEdit supports region-based editing with high-quality, automatically produced region annotations. Our experiments show that canonical diffusion-based editing baselines trained on UltraEdit set new records on challenging MagicBrush and Emu-Edit benchmarks, respectively. Our analysis further confirms the crucial role of real image anchors and region-based editing data. The dataset, code, and models will be made public.

TIST Journal 2023 Journal Article

Contrastive Learning Models for Sentence Representations

• Lingling Xu
• Haoran Xie
• Zongxi Li
• Fu Lee Wang
• Weiming Wang
• Qing Li

Sentence representation learning is a crucial task in natural language processing, as the quality of learned representations directly influences downstream tasks, such as sentence classification and sentiment analysis. Transformer-based pretrained language models such as bidirectional encoder representations from transformers (BERT) have been extensively applied to various natural language processing tasks, and have exhibited moderately good performance. However, the anisotropy of the learned embedding space prevents BERT sentence embeddings from achieving good results in the semantic textual similarity tasks. It has been shown that contrastive learning can alleviate the anisotropy problem and significantly improve sentence representation performance. Therefore, there has been a surge in the development of models that utilize contrastive learning to fine-tune BERT-like pretrained language models to learn sentence representations. But no systematic review of contrastive learning models for sentence representations has been conducted. To fill this gap, this article summarizes and categorizes the contrastive learning based sentence representation models, common evaluation tasks for assessing the quality of learned representations, and future research directions. Furthermore, we select several representative models for exhaustive experiments to illustrate the quantitative improvement of various strategies on sentence representations.

IJCAI Conference 2023 Conference Paper

Generative Diffusion Models on Graphs: Methods and Applications

• Chengyi Liu
• Wenqi Fan
• Yunqing Liu
• Jiatong Li
• Hang Li
• Hui Liu
• Jiliang Tang
• Qing Li

Diffusion models, as a novel generative paradigm, have achieved remarkable success in various image generation tasks such as image inpainting, image-to-text translation, and video generation. Graph generation is a crucial computational task on graphs with numerous real-world applications. It aims to learn the distribution of given graphs and then generate new graphs. Given the great success of diffusion models in image generation, increasing efforts have been made to leverage these techniques to advance graph generation in recent years. In this paper, we first provide a comprehensive overview of generative diffusion models on graphs, In particular, we review representative algorithms for three variants of graph diffusion models, i. e. , Score Matching with Langevin Dynamics (SMLD), Denoising Diffusion Probabilistic Model (DDPM), and Score-based Generative Model (SGM). Then, we summarize the major applications of generative diffusion models on graphs with a specific focus on molecule and protein modeling. Finally, we discuss promising directions in generative diffusion models on graph-structured data.

EAAI Journal 2023 Journal Article

Image segmentation of adhesive ores based on MSBA-Unet and convex-hull defect detection

• Wei Wang
• Qing Li
• Dezheng Zhang
• Jiawei Fu

Ore particle size information is a crucial indicator to evaluate the crushing quality and judge whether there are oversized ores on the conveyor belt. Accurately separating each ore is a critical prerequisite for obtaining high-precision particle size measurement (PSM) results. However, the large size variance and natural adhesion between ores pose a huge challenge to this task, imposing under-segmentation. Hence, this study proposes an automatic method that combines semantic segmentation and morphological operations to measure the ore particle size. Specifically, a novel multi-scale connection and boundary-aware U-Net model (MSBA-Unet) that classifies boundary pixels between adhesive ores more accurately is developed to segment ore images. Second, the convex-hull defect detection (CDD) method that divides the adhesive ores with a deep concave shape into two pieces is adopted to process the predicted masks further. The experimental results demonstrate that the MSBA-Unet architecture design and the CDD method can significantly improve the performance of separating adhesive ores of different sizes. Therefore, the under-segmentation problem is tremendously alleviated, and the ore PSM results agree well with the ground truth.

NeurIPS Conference 2023 Conference Paper

Interpreting Unsupervised Anomaly Detection in Security via Rule Extraction

• Ruoyu Li
• Qing Li
• Yu Zhang
• Dan Zhao
• Yong Jiang
• Yong Yang

Many security applications require unsupervised anomaly detection, as malicious data are extremely rare and often only unlabeled normal data are available for training (i. e. , zero-positive). However, security operators are concerned about the high stakes of trusting black-box models due to their lack of interpretability. In this paper, we propose a post-hoc method to globally explain a black-box unsupervised anomaly detection model via rule extraction. First, we propose the concept of distribution decomposition rules that decompose the complex distribution of normal data into multiple compositional distributions. To find such rules, we design an unsupervised Interior Clustering Tree that incorporates the model prediction into the splitting criteria. Then, we propose the Compositional Boundary Exploration (CBE) algorithm to obtain the boundary inference rules that estimate the decision boundary of the original model on each compositional distribution. By merging these two types of rules into a rule set, we can present the inferential process of the unsupervised black-box model in a human-understandable way, and build a surrogate rule-based model for online deployment at the same time. We conduct comprehensive experiments on the explanation of four distinct unsupervised anomaly detection models on various real-world datasets. The evaluation shows that our method outperforms existing methods in terms of diverse metrics including fidelity, correctness and robustness.

AAAI Conference 2023 Conference Paper

Joint Multimodal Entity-Relation Extraction Based on Edge-Enhanced Graph Alignment Network and Word-Pair Relation Tagging

• Li Yuan
• Yi Cai
• Jin Wang
• Qing Li

Multimodal named entity recognition (MNER) and multimodal relation extraction (MRE) are two fundamental subtasks in the multimodal knowledge graph construction task. However, the existing methods usually handle two tasks independently, which ignores the bidirectional interaction between them. This paper is the first to propose jointly performing MNER and MRE as a joint multimodal entity-relation extraction (JMERE) task. Besides, the current MNER and MRE models only consider aligning the visual objects with textual entities in visual and textual graphs but ignore the entity-entity relationships and object-object relationships. To address the above challenges, we propose an edge-enhanced graph alignment network and a word-pair relation tagging (EEGA) for the JMERE task. Specifically, we first design a word-pair relation tagging to exploit the bidirectional interaction between MNER and MRE and avoid error propagation. Then, we propose an edge-enhanced graph alignment network to enhance the JMERE task by aligning nodes and edges in the cross-graph. Compared with previous methods, the proposed method can leverage the edge information to auxiliary alignment between objects and entities and find the correlations between entity-entity relationships and object-object relationships. Experiments are conducted to show the effectiveness of our model.

JBHI Journal 2023 Journal Article

Jointly Fusing Multi-Scale Spatial-Logical Brain Networks: A Neural Decoding Method

• Ziyu Li
• Zhiyuan Zhu
• Qing Li
• Xia Wu

Functional magnetic resonance imaging (fMRI) is a methodology for measuring human brain activities. It has become more and more popular in neural decoding due to its noninvasive. Neural decoding aims to establishing models to reconstruct external stimuli or features of stimuli from known brain responses, so that we can understand the principles of brain functions such as emotion, cognition and language. Neural decoding based on fMRI is of great significance for further understanding the mechanism of brain operation. Most existing studies take multi-scale topology information of brain networks obtained from fMRI into account in neural decoding. However, they always ignore the simultaneous modeling of network structure and hemodynamic response, thus leading to information loss. In addition, current multi-scale methods usually only utilize spatial or logical reasoning relationship of brain networks, which brings challenge to precise neural decoding. In this work, we present a novel and robust multi-scale spatial and logical reasoning learning framework (MSLR) for fMRI-based neural decoding. Specifically, we first design graph signal wavelet generation module to combine brain network topology and node information to construct multi-scale representation of brain networks in a local to global manner. Then, we develop multi-scale information fusion module that can simultaneously model the spatial and logical reasoning relationship of brain networks, it can also learn discriminative multi-scale features with brain state transition. Finally, we construct a neural decoding module to predict the brain states. We evaluated the framework on the public Human Connectome Project (HCP) dataset that included 986 participants. The experimental results with support vector machine (SVM) outperform current state-of-the-art methods on four evaluation metrics (accuracy: 91. 58, kappa coefficient: 0. 883, macro F1: 0. 865 and hamming distance: 0. 105) under 19 different stimuli spanning 7 different cognitive tasks. The interpretation of the learned multi-scale representation replicates neuroscientific findings from previous fMRI studies and renews the multi-scale information flow pattern of brain network in neural decoding.

NeurIPS Conference 2023 Conference Paper

Learning non-Markovian Decision-Making from State-only Sequences

• Aoyang Qin
• Feng Gao
• Qing Li
• Song-Chun Zhu
• Sirui Xie

Conventional imitation learning assumes access to the actions of demonstrators, but these motor signals are often non-observable in naturalistic settings. Additionally, sequential decision-making behaviors in these settings can deviate from the assumptions of a standard Markov Decision Process (MDP). To address these challenges, we explore deep generative modeling of state-only sequences with non-Markov Decision Process (nMDP), where the policy is an energy-based prior in the latent space of the state transition generator. We develop maximum likelihood estimation to achieve model-based imitation, which involves short-run MCMC sampling from the prior and importance sampling for the posterior. The learned model enables $\textit{decision-making as inference}$: model-free policy execution is equivalent to prior sampling, model-based planning is posterior sampling initialized from the policy. We demonstrate the efficacy of the proposed method in a prototypical path planning task with non-Markovian constraints and show that the learned model exhibits strong performances in challenging domains from the MuJoCo suite.

NeurIPS Conference 2023 Conference Paper

Metis: Understanding and Enhancing In-Network Regular Expressions

• Zhengxin Zhang
• Yucheng Huang
• Guanglin Duan
• Qing Li
• Dan Zhao
• Yong Jiang
• Lianbo Ma
• Xi Xiao

Regular expressions (REs) offer one-shot solutions for many networking tasks, e. g. , network intrusion detection. However, REs purely rely on expert knowledge and cannot utilize labeled data for better accuracy. Today, neural networks (NNs) have shown superior accuracy and flexibility, thanks to their ability to learn from rich labeled data. Nevertheless, NNs are often incompetent in cold-start scenarios and too complex for deployment on network devices. In this paper, we propose Metis, a general framework that converts REs to network device affordable models for superior accuracy and throughput by taking advantage of REs' expert knowledge and NNs' learning ability. In Metis, we convert REs to byte-level recurrent neural networks (BRNNs) without training. The BRNNs preserve expert knowledge from REs and offer adequate accuracy in cold-start scenarios. When rich labeled data is available, the performance of BRNNs can be improved by training. Furthermore, we design a semi-supervised knowledge distillation to transform the BRNNs into pooling soft random forests (PSRFs) that can be deployed on network devices. To the best of our knowledge, this is the first method to employ model inference as an alternative to RE matching in network scenarios. We collect network traffic data on our campus for three weeks and evaluate Metis on them. Experimental results show that Metis is more accurate than original REs and other baselines, achieving superior throughput when deployed on network devices.

NeurIPS Conference 2023 Conference Paper

NeuralGF: Unsupervised Point Normal Estimation by Learning Neural Gradient Function

• Qing Li
• Huifang Feng
• Kanle Shi
• Yue Gao
• Yi Fang
• Yu-Shen Liu
• Zhizhong Han

Normal estimation for 3D point clouds is a fundamental task in 3D geometry processing. The state-of-the-art methods rely on priors of fitting local surfaces learned from normal supervision. However, normal supervision in benchmarks comes from synthetic shapes and is usually not available from real scans, thereby limiting the learned priors of these methods. In addition, normal orientation consistency across shapes remains difficult to achieve without a separate post-processing procedure. To resolve these issues, we propose a novel method for estimating oriented normals directly from point clouds without using ground truth normals as supervision. We achieve this by introducing a new paradigm for learning neural gradient functions, which encourages the neural network to fit the input point clouds and yield unit-norm gradients at the points. Specifically, we introduce loss functions to facilitate query points to iteratively reach the moving targets and aggregate onto the approximated surface, thereby learning a global surface representation of the data. Meanwhile, we incorporate gradients into the surface approximation to measure the minimum signed deviation of queries, resulting in a consistent gradient field associated with the surface. These techniques lead to our deep unsupervised oriented normal estimator that is robust to noise, outliers and density variations. Our excellent results on widely used benchmarks demonstrate that our method can learn more accurate normals for both unoriented and oriented normal estimation tasks than the latest methods. The source code and pre-trained model are publicly available.

TMLR Journal 2023 Journal Article

Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarios

• Xueying Zhan
• Zeyu Dai
• Qingzhong Wang
• Qing Li
• Haoyi Xiong
• Dejing Dou
• Antoni B. Chan

Pool-based Active Learning (AL) has proven successful in minimizing labeling costs by sequentially selecting the most informative unlabeled data from large pool and querying their labels from an oracle or annotators. However, existing AL sampling schemes may not perform well in out-of-distribution (OOD) data scenarios, where the unlabeled data pool contains samples that do not belong to the pre-defined categories of the target task. Achieving strong AL performance under OOD data scenarios presents a challenge due to the inherent conflict between AL sampling strategies and OOD data detection. For instance, both more informative in-distribution (ID) data and OOD data in an unlabeled data pool would be assigned high informativeness scores (e.g., high entropy) during AL processes. To address this dilemma, we propose a Monte-Carlo Pareto Optimization for Active Learning (POAL) sampling scheme, which selects optimal subsets of unlabeled samples with fixed batch size from the unlabeled data pool. We formulate the AL sampling task as a multi-objective optimization problem and employ Pareto optimization based on two conflicting objectives: (1) the conventional AL sampling scheme (e.g., maximum entropy) and (2) the confidence of excluding OOD data samples. Experimental results demonstrate the effectiveness of our POAL approach on classical Machine Learning (ML) and Deep Learning (DL) tasks.

TIST Journal 2023 Journal Article

Saliency Attack: Towards Imperceptible Black-box Adversarial Attack

• Zeyu Dai
• Shengcai Liu
• Qing Li
• Ke Tang

Deep neural networks are vulnerable to adversarial examples, even in the black-box setting where the attacker is only accessible to the model output. Recent studies have devised effective black-box attacks with high query efficiency. However, such performance is often accompanied by compromises in attack imperceptibility, hindering the practical use of these approaches. In this article, we propose to restrict the perturbations to a small salient region to generate adversarial examples that can hardly be perceived. This approach is readily compatible with many existing black-box attacks and can significantly improve their imperceptibility with little degradation in attack success rates. Furthermore, we propose the Saliency Attack, a new black-box attack aiming to refine the perturbations in the salient region to achieve even better imperceptibility. Extensive experiments show that compared to the state-of-the-art black-box attacks, our approach achieves much better imperceptibility scores, including most apparent distortion (MAD), L 0 and L 2 distances, and also obtains significantly better true success rate and effective query number judged by a human-like threshold on MAD. Importantly, the perturbations generated by our approach are interpretable to some extent. Finally, it is also demonstrated to be robust to different detection-based defenses.

NeurIPS Conference 2023 Conference Paper

SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models

• Hongxin Li
• Jingran Su
• Yuntao Chen
• Qing Li
• ZHAO-XIANG ZHANG

Computer end users have spent billions of hours completing daily tasks like tabular data processing and project timeline scheduling. Most of these tasks are repetitive and error-prone, yet most end users lack the skill to automate these burdensome works. With the advent of large language models (LLMs), directing software with natural language user requests become a reachable goal. In this work, we propose a SheetCopilot agent that takes natural language task and control spreadsheet to fulfill the requirements. We propose a set of atomic actions as an abstraction of spreadsheet software functionalities. We further design a state machine-based task planning framework for LLMs to robustly interact with spreadsheets. We curate a representative dataset containing 221 spreadsheet control tasks and establish a fully automated evaluation pipeline for rigorously benchmarking the ability of LLMs in software control tasks. Our SheetCopilot correctly completes 44. 3\% of tasks for a single generation, outperforming the strong code generation baseline by a wide margin. Our project page: https: //sheetcopilot. github. io/.

AAMAS Conference 2023 Conference Paper

User Device Interaction Prediction via Relational Gated Graph Attention Network and Intent-aware Encoder

• Jingyu Xiao
• Qingsong Zou
• Qing Li
• Dan Zhao
• Kang Li
• Wenxin Tang
• Runjie Zhou
• Yong Jiang

With the booming of smart home market, intelligent Internet of Things (IoT) devices have been increasingly more involved in home life. To improve the user experience of smart home, some prior works have explored how to use time series analysis technology for predicting the interaction between users and devices. However, existing solutions have inferior User Device Interaction (UDI) prediction accuracy, as they fail to consider the complex heterogeneous device transitions, multiple intents of a user and multi-level periodicity of user behaviors. In this paper, we present DeepUDI, a novel approach for accurate UDI prediction. First, we propose Relational Gated Graph Attention Network (RGGAT) to learn embedding of device and device control while considering complex heterogeneous temporal transitions. Second, we propose Intent-aware Encoder (IAE) to encode multiple intents of users via capsule networks. Third, we design a Historical Attention Mechanism (HAM) to capture the multi-level periodicity by aggregating the current sequence and the historical sequence representations through the attention mechanism. Comprehensive experiments on four realworld datasets show that DeepUDI consistently outperforms stateof-the-art baselines and also offers highly interpretable results.

NeurIPS Conference 2022 Conference Paper

Fairness Reprogramming

• Guanhua Zhang
• Yihua Zhang
• Yang Zhang
• Wenqi Fan
• Qing Li
• Sijia Liu
• Shiyu Chang

Despite a surge of recent advances in promoting machine Learning (ML) fairness, the existing mainstream approaches mostly require training or finetuning the entire weights of the neural network to meet the fairness criteria. However, this is often infeasible in practice for those large-scale trained models due to large computational and storage costs, low data efficiency, and model privacy issues. In this paper, we propose a new generic fairness learning paradigm, called FairReprogram, which incorporates the model reprogramming technique. Specifically, FairReprogram considers the case where models can not be changed and appends to the input a set of perturbations, called the fairness trigger, which is tuned towards the fairness criteria under a min-max formulation. We further introduce an information-theoretic framework that explains why and under what conditions fairness goals can be achieved using the fairness trigger. We show both theoretically and empirically that the fairness trigger can effectively obscure demographic biases in the output prediction of fixed ML models by providing false demographic information that hinders the model from utilizing the correct demographic information to make the prediction. Extensive experiments on both NLP and CV datasets demonstrate that our method can achieve better fairness improvements than retraining-based methods with far less data dependency under two widely-used fairness criteria. Codes are available at https: //github. com/UCSB-NLP-Chang/Fairness-Reprogramming. git.

NeurIPS Conference 2022 Conference Paper

HSurf-Net: Normal Estimation for 3D Point Clouds by Learning Hyper Surfaces

• Qing Li
• Yu-Shen Liu
• Jin-San Cheng
• Cheng Wang
• Yi Fang
• Zhizhong Han

We propose a novel normal estimation method called HSurf-Net, which can accurately predict normals from point clouds with noise and density variations. Previous methods focus on learning point weights to fit neighborhoods into a geometric surface approximated by a polynomial function with a predefined order, based on which normals are estimated. However, fitting surfaces explicitly from raw point clouds suffers from overfitting or underfitting issues caused by inappropriate polynomial orders and outliers, which significantly limits the performance of existing methods. To address these issues, we introduce hyper surface fitting to implicitly learn hyper surfaces, which are represented by multi-layer perceptron (MLP) layers that take point features as input and output surface patterns in a high dimensional feature space. We introduce a novel space transformation module, which consists of a sequence of local aggregation layers and global shift layers, to learn an optimal feature space, and a relative position encoding module to effectively convert point clouds into the learned feature space. Our model learns hyper surfaces from the noise-less features and directly predicts normal vectors. We jointly optimize the MLP weights and module parameters in a data-driven manner to make the model adaptively find the most suitable surface pattern for various points. Experimental results show that our HSurf-Net achieves the state-of-the-art performance on the synthetic shape dataset, the real-world indoor and outdoor scene datasets. The code, data and pretrained models are publicly available.

JBHI Journal 2022 Journal Article

Multiparametric Quantitative US Examination of Liver Fibrosis: A Feature-Engineering and Machine-Learning Based Analysis

• Huiying Wen
• Wei Zheng
• Min Li
• Qing Li
• Qiang Liu
• Jianhua Zhou
• Zhong Liu
• Xin Chen

Quantitative ultrasound (QUS), which attempts to extract quantitative features from the US radiofrequency (RF) or envelope data for tissue characterization, is becoming a promising technique for noninvasive assessments of liver fibrosis. However, the number of feature variables examined and finally used in the existing QUS methods is typically small, limiting the diagnostic performance. Therefore, this paper devises a new multiparametric QUS (MP-QUS) method which enables the extraction of a large number of feature variables from US RF signals and allows for the use of feature-engineering and machine-learning based algorithms for liver fibrosis assessment. In the MP-QUS, eighty-four feature variables were extracted from multiple QUS parametric maps derived from the RF signals and the envelope data. Afterwards, feature reduction and selection were performed in turn to remove the feature redundancy and identify the best combination of features in the reduced feature set. Finally, a variety of machine-learning algorithms were tested for fibrosis classification with the selected features, based on the results of which the optimal classifier was established. The performance of the proposed MP-QUS method for staging liver fibrosis was evaluated on an animal model, with histologic examination as the reference standard. The mean accuracy, sensitivity, specificity and area under the receiver-operating-characteristic curve achieved by MP-QUS are respectively 83. 38%, 86. 04%, 80. 82%, and 0. 891 for recognizing significant liver fibrosis, and 85. 50%, 88. 92%, 85. 24%, and 0. 924 for diagnosing liver cirrhosis. The proposed MP-QUS method paves a way for its future extension to assess liver fibrosis in human subjects.

EAAI Journal 2022 Journal Article

Robust self-supervised monocular visual odometry based on prediction-update pose estimation network

• Haixin Xiu
• Yiyou Liang
• Hui Zeng
• Qing Li
• Hongmin Liu
• Bin Fan
• Chen Li

Visual odometry aims at estimating the camera pose from video sequence, which is an important part of visual Simultaneous Localization and Mapping (SLAM). In this paper, we propose a novel prediction-update pose estimation network, PU-PoseNet, for self-supervised monocular visual odometry. It allows the network to use the effective information of the previous frame in estimating the current pose. The long-time pose consistency constraint-based motion weighted photometric loss is designed to make the network to pay more attention to the pixels of stationary objects and enhance the time consistency of estimation results. The depth image-based occlusion detection, the depth smoothness loss and auto-Masking are used to construct the depth consistency constraint loss term to reduce the influences of interferences such as occlusion. To further improve the robustness and the accuracy of the proposed method, both the depth consistency constraint and the variational auto-encoder are used for network training. For frame missing cases, a novel frame missing training strategy is used to make our method adapt frame missing cases. Extensive experiments on the KITTI dataset have validated the effectiveness of our proposed method.

IJCAI Conference 2022 Conference Paper

Subsequence-based Graph Routing Network for Capturing Multiple Risk Propagation Processes

• Rui Cheng
• Qing Li

In finance, the risk of an entity depends not only on its historical information but also on the risk propagated by its related peers. Pilot studies rely on Graph Neural Networks (GNNs) to model this risk propagation, where each entity is treated as a node and represented by its time-series information. However, conventional GNNs are constrained by their unified messaging mechanism with an assumption that the risk of a given entity only propagates to its related peers with the same time lag and has the same effect, which is against the ground truth. In this study, we propose the subsequence-based graph routing network (S-GRN) for capturing the variant risk propagation processes among different time-series represented entities. In S-GRN, the messaging mechanism between each node pair is dynamically and independently selected from multiple messaging mechanisms based on the dependencies of variant subsequence patterns. The S-GRN is extensively evaluated on two synthetic tasks and three real-world datasets and demonstrates state-of-the-art performance.

IJCAI Conference 2021 Conference Paper

A Comparative Survey: Benchmarking for Pool-based Active Learning

• Xueying Zhan
• Huan Liu
• Qing Li
• Antoni B. Chan

Active learning (AL) is a subfield of machine learning (ML) in which a learning algorithm aims to achieve good accuracy with fewer training samples by interactively querying the oracles to label new data points. Pool-based AL is well-motivated in many ML tasks, where unlabeled data is abundant, but their labels are hard or costly to obtain. Although many pool-based AL methods have been developed, some important questions remain unanswered such as how to: 1) determine the current state-of-the-art technique; 2) evaluate the relative benefit of new methods for various properties of the dataset; 3) understand what specific problems merit greater attention; and 4) measure the progress of the field over time. In this paper, we survey and compare various AL strategies used in both recently proposed and classic highly-cited methods. We propose to benchmark pool-based AL methods with a variety of datasets and quantitative metric, and draw insights from the comparative empirical results.

JBHI Journal 2021 Journal Article

Altered Time-Frequency Feature in Default Mode Network of Autism Based on Improved Hilbert-Huang Transform

• Han Zhang
• Rui Li
• Xiaotong Wen
• Qing Li
• Xia Wu

Autism spectrum disorder (ASD) is a pervasive neurodevelopmental disorder characterized by restricted interests and repetitive behaviors. Non-invasive measurements of brain activity with functional magnetic resonance imaging (fMRI) have demonstrated that the abnormality in the default mode network (DMN) is a crucial neural basis of ASD, but the time-frequency feature of the DMN has not yet been revealed. Hilbert-Huang transform (HHT) is conducive to feature extraction of biomedical signals and has recently been suggested as an effective way to explore the time-frequency feature of the brain mechanism. In this study, the resting-state fMRI dataset of 105 subjects including 59 ASD participants and 46 healthy control (HC) participants were involved in the time-frequency clustering analysis based on improved HHT and modified k-means clustering with label-replacement. Compared with HC, ASD selectively showed enhanced Hilbert weight frequency (HWF) in high frequency bands in crucial regions of the DMN, including the medial prefrontal cortex (MPFC), posterior cingulate cortex (PCC) and anterior cingulate cortex (ACC). Time-frequency clustering analysis revealed altered DMN organization in ASD. In the posterior DMN, the PCC and bilateral precuneus were separated for HC but clustered for ASD; in the anterior DMN, the clusters of ACC, dorsal MPFC, and ventral MPFC were relatively scattered for ASD. This study paves a promising way to uncover the alteration in the DMN and identifies a potential neuroimaging biomarker of diagnostic reference for ASD.

AAAI Conference 2021 Short Paper

An Entity-Aware Adversarial Domain Adaptation Network for Cross-Domain Named Entity Recognition (Student Abstract)

• Qi Peng
• Changmeng Zheng
• Yi Cai
• Tao Wang
• Haoran Xie
• Qing Li

Existing methods for named entity recognition are critically relied on labeled data. To handle the situation that the data is fully-unlabeled, we propose an entity-aware adversarial domain adaptation network, which utilizes the labeled source data and then adapts to unlabeled target domain. We first apply adversarial training to reduce the distribution gap between different domains. Furthermore, we introduce an entity-aware attention to guide adversarial process to achieve the alignment of entity features. The experiment shows that our model outperforms the state-of-the-art approaches.

EAAI Journal 2021 Journal Article

Counting trees with point-wise supervised segmentation network

• Pinmo Tong
• Pengcheng Han
• Suicheng Li
• Ni Li
• Shuhui Bu
• Qing Li
• Ke Li

Tree counting plays an important role in wide applications of environmental protection, agricultural planning and crop yield estimation. However, traditional tree counting methods require expensive feature engineering, which causes additional mistake and cannot be optimized overall. Recently, deep learning based approaches have been adopted for this task which demonstrate state-of-the-art performance. In this paper, a point-wise supervised segmentation network is proposed based on a deep segmentation network with only weak supervision, which can complete localization and generate mask of each tree simultaneously. In the first step, a tree feature extractor module is adopted to extract features of input images with a novel encoder–decoder network. In the second step, an effective strategy is designed to deal with different conditions with mask predictions. Finally, the basic localization and rectification guidance are introduced to train the whole network. In addition, two different datasets are created and an existing challenging plant dataset is selected to evaluate the proposed method. Experimental results on those datasets show that the proposed method outperforms the state-of-the-art methods in most challenging conditions. This method has great potential to reduce human labor due to effective automatic generated masks.

AAAI Conference 2021 Conference Paper

Entity Guided Question Generation with Contextual Structure and Sequence Information Capturing

• Qingbao Huang
• Mingyi Fu
• Linzhang Mo
• Yi Cai
• Jingyun Xu
• Pijian Li
• Qing Li
• Ho-fung Leung

Question generation is a challenging task and has attracted widespread attention in recent years. Although previous studies have made great progress, there are still two main shortcomings: First, previous work did not simultaneously capture the sequence information and structure information hidden in the context, which results in poor results of the generated questions. Second, the generated questions cannot be answered by the given context. To tackle these issues, we propose an entity guided question generation model with contextual structure information and sequence information capturing. We use a Graph Convolutional Network and a Bidirectional Long Short Term Memory Network to capture the structure information and sequence information of the context, simultaneously. In addition, to improve the answerability of the generated questions, we use an entity-guided approach to obtain question type from the answer, and jointly encode the answer and question type. Both automatic and manual metrics show that our model can generate comparable questions with state-of-the-art models. Our code is available at https: //github. com/VISLANG-Lab/EGSS.

EAAI Journal 2021 Journal Article

HK–SEIR model of public opinion evolution based on communication factors

• Qing Li
• YaJun Du
• ZhaoYan Li
• JinRong Hu
• RuiLin Hu
• BingYan Lv
• Peng Jia

Microblog, with its good interaction and convenient dissemination, has become the main platform for public opinion dissemination. How to discover the law of public opinion dissemination, and to identify the public opinion accurately have become the hot researches. In this paper, we define the user influence, topic popularity, topic interest to analysis the process of opinions fusion among the users under the interest and confidence threshold. We propose a new public opinion evolution HK–SEIR model which combines the opinion fusion HK and the epidemic transmission SEIR models. Firstly, the topic interest degree is added to the opinion fusion HK model, and the interaction behavior between the users under the interest and confidence threshold is analyzed. Then, we calculate the probability of topic propagation caused by the interaction of opinions between users under group pressure, and the probability that users change from the infected state to the removed state under topic popularity. Finally, we analyze the changes of the susceptible, exposed, infected and removed states in the process of public opinion communication. The experiment proves that the HK–SEIR model is closer to the work-rest rules of public opinion communication than SEIR, SIR model. The density peak time is closer to the peak of real public opinion communication. We find that the user interest is the main factor influencing the public opinion dissemination after the interaction of user opinions fusion reaches a certain degree. The negative public opinion of the higher proportion can easily reach the peak of public opinion propagation.

EAAI Journal 2021 Journal Article

HMMN: Online metric learning for human re-identification via hard sample mining memory network

• Pengcheng Han
• Qing Li
• Cunbao Ma
• Shibiao Xu
• Shuhui Bu
• Yong Zhao
• Ke Li

Effective metric learning is important in various applications, especially for re-identification. Compared with most existing re-identification methods which are not suitable for a real-time update mode, we exploit a novel memory-based strategy for mining hard triplets in online metric learning. This strategy is realized with an end-to-end deep learning based framework using an external memory pool. Our proposed pipeline is able to explicitly provide hard negative and positive samples to generate effective triplets, which are important for online metric learning due to the representative triplets could provide distinctive information to help understand the concept of metric learning between categories. In addition, a “focal-triplet loss” function is proposed to deal with the lack of positive or negative samples for one anchor, and the imbalance between easy and hard triplets for mini-batch. Experimental results on Market-1501, CUHK03 and DukeMTMC-reID demonstrate the effectiveness of our method, and its performance even outperforms that of some existing offline methods.

AAAI Conference 2021 Conference Paper

Learning by Fixing: Solving Math Word Problems with Weak Supervision

• Yining Hong
• Qing Li
• Daniel Ciao
• Siyuan Huang
• Song-Chun Zhu

Previous neural solvers of math word problems (MWPs) are learned with full supervision and fail to generate diverse solutions. In this paper, we address this issue by introducing a weakly-supervised paradigm for learning MWPs. Our method only requires the annotations of the final answers and can generate various solutions for a single problem. To boost weaklysupervised learning, we propose a novel learning-by-fixing (LBF) framework, which corrects the misperceptions of the neural network via symbolic reasoning. Specifically, for an incorrect solution tree generated by the neural network, the fixing mechanism propagates the error from the root node to the leaf nodes and infers the most probable fix that can be executed to get the desired answer. To generate more diverse solutions, tree regularization is applied to guide the efficient shrinkage and exploration of the solution space, and a memory buffer is designed to track and save the discovered various fixes for each problem. Experimental results on the Math23K dataset show the proposed LBF framework significantly outperforms reinforcement learning baselines in weakly-supervised learning. Furthermore, it achieves comparable top-1 and much better top-3/5 answer accuracies than fully-supervised methods, demonstrating its strength in producing diverse solutions.

AAAI Conference 2021 Conference Paper

Merging Statistical Feature via Adaptive Gate for Improved Text Classification

• Xianming Li
• Zongxi Li
• Haoran Xie
• Qing Li

Currently, text classification studies mainly focus on training classifiers by using textual input only, or enhancing semantic features by introducing external knowledge (e. g. , hand-craft lexicons and domain knowledge). In contrast, some intrinsic statistical features of the corpus, like word frequency and distribution over labels, are not well exploited. Compared with external knowledge, the statistical features are deterministic and naturally compatible with corresponding tasks. In this paper, we propose an Adaptive Gate Network (AGN) to consolidate semantic representation with statistical features selectively. In particular, AGN encodes statistical features through a variational component and merges information via a welldesigned valve mechanism. The valve adapts the information flow into the classifier according to the confidence of semantic features in decision making, which can facilitate training a robust classifier and can address the overfitting caused by using statistical features. Extensive experiments on datasets of various scales show that, by incorporating statistical information, AGN can improve the classification performance of CNN, RNN, Transformer, and Bert based models effectively. The experiments also indicate the robustness of AGN against adversarial attacks of manipulating statistical information.

AAAI Conference 2021 Conference Paper

Modeling the Momentum Spillover Effect for Stock Prediction via Attribute-Driven Graph Attention Networks

• Rui Cheng
• Qing Li

In finance, the momentum spillovers of listed firms is well acknowledged. Only few studies predicted the trend of one firm in terms of its relevant firms. A common strategy of the pilot work is to adopt graph convolution networks (GCNs) with some predefined firm relations. However, momentum spillovers are propagated via a variety of firm relations, of which the bridging importance varies with time. Restricting to several predefined relations inevitably makes noise and thus misleads stock predictions. In addition, traditional GCNs transfer and aggregate the peer influences without considering the states of both connected firms once a connection is built. Such non-attribute sensibility makes traditional GCNs inappropriate to deal with the attribute-sensitive momentum spillovers of listed firms wherein the abnormal price drop of one firm may not spill over if the trade volume of this decreasing price is small or the prices of the linked firms are undervalued. In this study, we propose an attribute-driven graph attention network (AD-GAT) to address both problems in modeling momentum spillovers. This is achieved by element-wisely multiplying the nonlinear transformation of the attributes of the connected firms with the attributes of the source firm to consider its attribute-sensitive momentum spillovers, and applying the unmasked attention mechanism to infer the general dynamic firm relation from observed market signals fused by a novel tensor-based feature extractor. Experiments on the three-year data of the S&P 500 demonstrate the superiority of the proposed framework over stateof-the-art algorithms, including GCN, eLSTM, and TGC.

AIIM Journal 2021 Journal Article

MvKFN-MDA: Multi-view Kernel Fusion Network for miRNA-disease association prediction

• Jin Li
• Tao Liu
• Jingru Wang
• Qing Li
• Chenxi Ning
• Yun Yang

Predicting the associations between microRNAs (miRNAs) and diseases is of great significance for identifying miRNAs related to human diseases. Since it is time-consuming and costly to identify the association between miRNA and disease through biological experiments, computational methods are currently used as an effective supplement to identify the potential association between disease and miRNA. This paper presents a Multi-view Kernel Fusion Network (MvKFN) based prediction method (MvKFN-MDA) to address the problem of miRNA-disease associations prediction. A novel multiple kernel fusion framework Multi-view Kernel Fusion Network (MvKFN) is first proposed to effectively fuse different views similarity kernels constructed from different data sources in a highly nonlinear way. Using MvKFNs, both different base similarity kernels for miRNA, such as sequence, functional, semantic, Gaussian profile kernels and different base similarity kernels for diseases, such as semantic, Gaussian profile kernel are nonlinearly fused into two integrated similarity kernels, one for miRNA, another for disease. Then, miRNA and disease feature representations are extracted from the miRNA and disease integrated similarity kernels respectively. These features are then fed into a neural matrix completion framework which finally outputs the association prediction scores. The parameters of MvKFN-MDA are learned based on the known miRNA-disease association matrix in a supervised end-to-end way. We compare the proposed method with other state-of-the-art methods. The AUCs of our proposed method were superior to the existing methods in both 5-FCV and LOOCV on two open experimental datasets. Furthermore, 49, 48, and 47 of the top 50 predicted miRNAs for three high-risk human diseases, namely, colon cancer, lymphoma, and kidney cancer, are verified respectively using experimental literature. Finally, 100% accuracy from the top 50 predicted miRNAs is achieved when breast cancer is used as a case study to evaluate the ability of MvKFN-MDA for predicting a new disease without any known related miRNAs.

AAAI Conference 2021 Conference Paper

SMART: A Situation Model for Algebra Story Problems via Attributed Grammar

• Yining Hong
• Qing Li
• Ran Gong
• Daniel Ciao
• Siyuan Huang
• Song-Chun Zhu

Solving algebra story problems remains a challenging task in artificial intelligence, which requires a detailed understanding of real-world situations and a strong mathematical reasoning capability. Previous neural solvers of math word problems directly translate problem texts into equations, lacking an explicit interpretation of the situations, and often fail to handle more sophisticated situations. To address such limits of neural solvers, we introduce the concept of a situation model, which originates from psychology studies to represent the mental states of humans in problem-solving, and propose SMART, which adopts attributed grammar as the representation of situation models for algebra story problems. Specifically, we first train an information extraction module to extract nodes, attributes, and relations from problem texts and then generate a parse graph based on a pre-defined attributed grammar. An iterative learning strategy is also proposed to improve the performance of SMART further. To rigorously study this task, we carefully curate a new dataset named ASP6. 6k. Experimental results on ASP6. 6k show that the proposed model outperforms all previous neural solvers by a large margin while preserving much better interpretability. To test these models’ generalization capability, we also design an out-of-distribution (OOD) evaluation, in which problems are more complex than those in the training set. Our model exceeds state-of-the-art models by 17% in the OOD evaluation, demonstrating its superior generalization ability.

AAAI Conference 2021 Conference Paper

Story Ending Generation with Multi-Level Graph Convolutional Networks over Dependency Trees

• Qingbao Huang
• Linzhang Mo
• Pijian Li
• Yi Cai
• Qingguang Liu
• Jielong Wei
• Qing Li
• Ho-fung Leung

As an interesting and challenging task, story ending generation aims at generating a reasonable and coherent ending for a given story context. The key challenge of the task is to comprehend the context sufficiently and capture the hidden logic information effectively, which has not been well explored by most existing generative models. To tackle this issue, we propose a context-aware Multi-level Graph Convolutional Networks over Dependency Parse (MGCN-DP) trees to capture dependency relations and context clues more effectively. We utilize dependency parse trees to facilitate capturing relations and events in the context implicitly, and Multilevel Graph Convolutional Networks to update and deliver the representation crossing levels to obtain richer contextual information. Both automatic and manual evaluations show that our MGCN-DP can achieve comparable performance with state-of-the-art models. Our source code is available at https: //github. com/VISLANG-Lab/MLGCN-DP.

IJCAI Conference 2021 Conference Paper

Tracklet Proposal Network for Multi-Object Tracking on Point Clouds

• Hai Wu
• Qing Li
• Chenglu Wen
• Xin Li
• Xiaoliang Fan
• Cheng Wang

This paper proposes the first tracklet proposal network, named PC-TCNN, for Multi-Object Tracking (MOT) on point clouds. Our pipeline first generates tracklet proposals, then refines these tracklets and associates them to generate long trajectories. Specifically, object proposal generation and motion regression are first performed on a point cloud sequence to generate tracklet candidates. Then, spatial-temporal features of each tracklet are exploited and their consistency is used to refine the tracklet proposal. Finally, the refined tracklets across multiple frames are associated to perform MOT on the point cloud sequence. The PC-TCNN significantly improves the MOT performance by introducing the tracklet proposal design. On the KITTI tracking benchmark, it attains an MOTA of 91. 75%, outperforming all submitted results on the online leaderboard.

EAAI Journal 2020 Journal Article

A reaction–diffusion based level set method for image segmentation in three dimensions

• Zhe Zhang
• Yi Min Xie
• Qing Li
• Shiwei Zhou

The image segmentation of computed tomography data for three-dimensional biological structures remains challenging because of the limitations of existing numerical techniques and computer resources. The work represents the structures as the zero-level contour of a level set function whose value is constrained to a narrow band ranging. A cost functional composed of fitting energy for extracting the local intensity and diffusion energy for regularization is minimized within a framework of optimization. To avoid the re-initialization procedure and accelerate the convergence when updating the level set function, a reaction–diffusion technique is developed to replace the upwind algorithm by finite element analysis. Numerical examples demonstrate elegant biological structures with clear and smooth interfaces can be generated within a few iteration steps because the time step 100-fold larger than the allowable value of Courant–Friedrichs–Lewy stability condition can be applied in the proposed method.

IJCAI Conference 2020 Conference Paper

Bridging Cross-Tasks Gap for Cognitive Assessment via Fine-Grained Domain Adaptation

• Yingwei Zhang
• Yiqiang Chen
• Hanchao Yu
• Zeping Lv
• Qing Li
• Xiaodong Yang

Discriminating pathologic cognitive decline from the expected decline of normal aging is an important research topic for elderly care and health monitoring. However, most cognitive assessment methods only work when data distributions of the training set and testing set are consistent. Enabling existing cognitive assessment models to adapt to the data in new cognitive assessment tasks is a significant challenge. In this paper, we propose a novel domain adaptation method, namely the Fine-Grained Adaptation Random Forest (FAT), to bridge the cognitive assessment gap when the data distribution is changed. FAT is composed of two essential parts 1) information gain based model evaluation strategy (IGME) and 2) domain adaptation tree growing mechanism (DATG). IGME is used to evaluate every individual tree, and DATG is used to transfer the source model to the target domain. To evaluate the performance of FAT, we conduct experiments in real clinical environments. Experimental results demonstrate that FAT is significantly more accurate and efficient compared with other state-of-the-art methods.

EAAI Journal 2020 Journal Article

Change detection in images using shape-aware siamese convolutional network

• Suicheng Li
• Pengcheng Han
• Shuhui Bu
• Pinmo Tong
• Qing Li
• Ke Li
• Gang Wan

Change detection gradually becomes a core technique due to its wide applications of image or video analysis like land cover analysis and real-time monitoring system. Recently, siamese convolutional networks have been adopted for change detection which demonstrate the state-of-the-art performance. Although most of the previous works have better location accuracy, these methods cannot avoid side effects such as coarse boundaries and empty holes. In this paper, we propose a shape-aware siamese convolutional network (SASCNet) to simultaneously integrate different information for change detection with three steps in an unified network. In the first step, we extract multi-dimension features from paired images and select multi-level change maps generated by a novel siamese encoder–decoder network with multi-scale supervisions. In the second step, we integrate these change maps to obtain complementary information in detail. Finally, we use a residual fine-tune module to refine the predicted change maps and enhance the performance. Because of rich information in different levels and multi-scale supervisions, the predicted change maps could provide precise positioning as well as high-quality shapes. Experimental results on “CDnet 2014 dataset” and “AICD-2012 dataset” show that our method outperforms the state-of-the-art methods in most challenging conditions.

IS Journal 2020 Journal Article

Personalized Geographical Influence Modeling for POI Recommendation

• Yanan Zhang
• Guanfeng Liu
• An Liu
• Yifan Zhang
• Zhixu Li
• Xiangliang Zhang
• Qing Li

Point-of-interest (POI) recommendation has great significance in helping users find favorite places from a large number of candidate venues. One challenging in POI recommendation is to effectively exploit geographical information since users usually care about the physical distance to the recommended POIs. Though spatial relevance has been widely considered in recent recommendation methods, it is modeled only from the POI perspective, failing to capture user personalized preference to spatial distance. Moreover, these methods suffer from a diversity-deficiency problem since they are often based on collaborative filtering which always favors popular POIs. To overcome these problems, we propose in this article a personalized geographical influence modeling method called PGIM, which jointly learns users’ geographical preference and diversity preference for POI recommendation. Specifically, we model geographical preference from three aspects: user global tolerance, user local tolerance, and spatial distance. We also extract user diversity preference from interactions among users for diversity-promoting recommendation. Experimental results on three real-world datasets demonstrate the superiority of PGIM.

AAAI Conference 2020 Conference Paper

Point2Node: Correlation Learning of Dynamic-Node for Point Cloud Feature Modeling

• Wenkai Han
• Chenglu Wen
• Cheng Wang
• Xin Li
• Qing Li

Fully exploring correlation among points in point clouds is essential for their feature modeling. This paper presents a novel end-to-end graph model, named Point2Node, to represent a given point cloud. Point2Node can dynamically explore correlation among all graph nodes from different levels, and adaptively aggregate the learned features. Specifically, first, to fully explore the spatial correlation among points for enhanced feature description, in a high-dimensional node graph, we dynamically integrate the node’s correlation with self, local, and non-local nodes. Second, to more effectively integrate learned features, we design a data-aware gate mechanism to self-adaptively aggregate features at the channel level. Extensive experiments on various point cloud benchmarks demonstrate that our method outperforms the state-ofthe-art.

IJCAI Conference 2019 Conference Paper

Deep Adversarial Social Recommendation

• Wenqi Fan
• Tyler Derr
• Yao Ma
• Jianping Wang
• Jiliang Tang
• Qing Li

Recent years have witnessed rapid developments on social recommendation techniques for improving the performance of recommender systems due to the growing influence of social networks to our daily life. The majority of existing social recommendation methods unify user representation for the user-item interactions (item domain) and user-user connections (social domain). However, it may restrain user representation learning in each respective domain, since users behave and interact differently in the two domains, which makes their representations to be heterogeneous. In addition, most of traditional recommender systems can not efficiently optimize these objectives, since they utilize negative sampling technique which is unable to provide enough informative guidance towards the training during the optimization process. In this paper, to address the aforementioned challenges, we propose a novel deep adversarial social recommendation framework DASO. It adopts a bidirectional mapping method to transfer users' information between social domain and item domain using adversarial learning. Comprehensive experiments on two real-world datasets show the effectiveness of the proposed framework.

AAAI Conference 2018 Short Paper

Deep Modeling of Social Relations for Recommendation

• Wenqi Fan
• Qing Li
• Min Cheng

Social-based recommender systems have been recently proposed by incorporating social relations of users to alleviate sparsity issue of user-to-item rating data and to improve recommendation performance. Many of these social-based recommender systems linearly combine the multiplication of social features between users. However, these methods lack the ability to capture complex and intrinsic non-linear features from social relations. In this paper, we present a deep neural network based model to learn non-linear features of each user from social relations, and to integrate into probabilistic matrix factorization for rating prediction problem. Experiments demonstrate the advantages of the proposed method over stateof-the-art social-based recommender systems.

IJCAI Conference 2018 Conference Paper

Unpaired Multi-Domain Image Generation via Regularized Conditional GANs

• Xudong Mao
• Qing Li

In this paper, we study the problem of multi-domain image generation, the goal of which is to generate pairs of corresponding images from different domains. With the recent development in generative models, image generation has achieved great progress and has been applied to various computer vision tasks. However, multi-domain image generation may not achieve the desired performance due to the difficulty of learning the correspondence of different domain images, especially when the information of paired samples is not given. To tackle this problem, we propose Regularized Conditional GAN (RegCGAN) which is capable of learning to generate corresponding images in the absence of paired training data. RegCGAN is based on the conditional GAN, and we introduce two regularizers to guide the model to learn the corresponding semantics of different domains. We evaluate the proposed model on several tasks for which paired training data is not given, including the generation of edges and photos, the generation of faces with different attributes, etc. The experimental results show that our model can successfully generate corresponding images for all these tasks, while outperforms the baseline methods. We also introduce an approach of applying RegCGAN to unsupervised domain adaptation.

IJCAI Conference 2017 Conference Paper

A Robust Noise Resistant Algorithm for POI Identification from Flickr Data

• Yiyang Yang
• Zhiguo Gong
• Qing Li
• Leong Hou U
• Ruichu Cai
• Zhifeng Hao

Point of Interests (POI) identification using social media data (e. g. Flickr, Microblog) is one of the most popular research topics in recent years. However, there exist large amounts of noises (POI irrelevant data) in such crowd-contributed collections. Traditional solutions to this problem is to set a global density threshold and remove the data point as noise if its density is lower than the threshold. However, the density values vary significantly among POIs. As the result, some POIs with relatively lower density could not be identified. To solve the problem, we propose a technique based on the local drastic changes of the data density. First we define the local maxima of the density function as the Urban POIs, and the gradient ascent algorithm is exploited to assign data points into different clusters. To remove noises, we incorporate the Laplacian Zero-Crossing points along the gradient ascent process as the boundaries of the POI. Points located outside the POI region are regarded as noises. Then the technique is extended into the geographical and textual joint space so that it can make use of the heterogeneous features of social media. The experimental results show the significance of the proposed approach in removing noises.

IS Journal 2015 Journal Article

Does Summarization Help Stock Prediction? A News Impact Analysis

• Xiaodong Li
• Haoran Xie
• Yangqiu Song
• Shanfeng Zhu
• Qing Li
• Fu Lee Wang

The authors study the problem of how news summarization can help stock price prediction, proposing a generic stock price prediction framework to enable the use of different external signals to predict stock prices. Experiments were conducted on five years of Hong Kong Stock Exchange data, with news reported by Finet; evaluations were performed at individual stock, sector index, and market index levels. The authors' results show that prediction based on news article summarization can effectively outperform prediction based on full-length articles on both validation and independent testing sets.

AAAI Conference 2015 Conference Paper

Tensor-Based Learning for Predicting Stock Movements

• Qing Li
• LiLing Jiang
• Ping Li
• Hsinchun Chen

Stock movements are essentially driven by new information. Market data, financial news, and social sentiment are believed to have impacts on stock markets. To study the correlation between information and stock movements, previous works typically concatenate the features of different information sources into one super feature vector. However, such concatenated vector approaches treat each information source separately and ignore their interactions. In this article, we model the multi-faceted investors’ information and their intrinsic links with tensors. To identify the nonlinear patterns between stock movements and new information, we propose a supervised tensor regression learning approach to investigate the joint impact of different information sources on stock markets. Experiments on CSI 100 stocks in the year 2011 show that our approach outperforms the state-of-the-art trading strategies.

JBHI Journal 2014 Journal Article

Brain CT Image Similarity Retrieval Method Based on Uncertain Location Graph

• Haiwei Pan
• Pengyuan Li
• Qing Li
• Qilong Han
• Xiaoning Feng
• Linlin Gao

A number of brain computed tomography (CT) images stored in hospitals that contain valuable information should be shared to support computer-aided diagnosis systems. Finding the similar brain CT images from the brain CT image database can effectively help doctors diagnose based on the earlier cases. However, the similarity retrieval for brain CT images requires much higher accuracy than the general images. In this paper, a new model of uncertain location graph (ULG) is presented for brain CT image modeling and similarity retrieval. According to the characteristics of brain CT image, we propose a novel method to model brain CT image to ULG based on brain CT image texture. Then, a scheme for ULG similarity retrieval is introduced. Furthermore, an effective index structure is applied to reduce the searching time. Experimental results reveal that our method functions well on brain CT images similarity retrieval with higher accuracy and efficiency.

TIST Journal 2013 Journal Article

Introduction to the special section on intelligent tutoring and coaching systems

• Qing Li
• Xiangfeng Luo
• Liu Wenyin
• Cristina Conati

YNIMG Journal 2010 Journal Article

Longitudinal characterization of brain atrophy of a Huntington's disease mouse model by automated morphological analyses of magnetic resonance images

• Jiangyang Zhang
• Qi Peng
• Qing Li
• Neda Jahanshad
• Zhipeng Hou
• Mali Jiang
• Naoki Masuda
• Douglas R. Langbehn

Mouse models of human diseases play crucial roles in understanding disease mechanisms and developing therapeutic measures. Huntington's disease (HD) is characterized by striatal atrophy that begins long before the onset of motor symptoms. In symptomatic HD, striatal volumes decline predictably with disease course. Thus, imaging based volumetric measures have been proposed as outcomes for presymptomatic as well as symptomatic clinical trials of HD. Magnetic resonance imaging of the mouse brain structures is becoming widely available and has been proposed as one of the biomarkers of disease progression and drug efficacy testing. However, three-dimensional and quantitative morphological analyses of the brains are not straightforward. In this paper, we describe a tool for automated segmentation and voxel-based morphological analyses of the mouse brains. This tool was applied to a well-established mouse model of Huntington's disease, the R6/2 transgenic mouse strain. Comparison between the automated and manual segmentation results showed excellent agreement in most brain regions. The automated method was able to sensitively detect atrophy as early as 4 weeks of age and accurately follow disease progression. Comparison between ex vivo and in vivo MRI suggests that the ex vivo end-point measurement of brain morphology is also a valid approach except for the morphology of the ventricles. This is the first report of longitudinal characterization of brain atrophy in a mouse model of Huntington's disease by using automatic morphological analysis.

AAAI Conference 2010 Conference Paper

News Recommendation in Forum-Based Social Media

• Jia Wang
• Qing Li
• Yuanzhu Chen
• Jiafen Liu
• Chen Zhang
• Zhangxi Lin

Self-publication of news on Web sites is becoming a common application platform to enable more engaging interaction among users. Discussion in the form of comments following news postings can be effectively facilitated if the service provider can recommend articles based on not only the original news itself but also the thread of changing comments. This turns the traditional news recommendation to a “discussion moderator” that can intelligently assist online forums. In this work, we present a framework to implement such adaptive news recommendation. In addition, to alleviate the problem of recommending essentially identical articles, the relationship (duplication, generalization or specialization) between suggested news articles and the original posting is investigated. Experiments indicate that our proposed solutions provide an enhanced news recommendation service in forum-based social media.

AAAI Conference 2008 Conference Paper

Extracting Relevant Snippets for Web Navigation

• Qing Li

IS Journal 2004 Journal Article

Toward intelligent driver-assistance and safety warning system

• Nan-Ning Zheng
• Shuming Tang
• Hong Cheng
• Qing Li
• G. Lai
• F.-W. Wang

A major problem associated with the rapid growth in automotive production is an increase in traffic congestion and accidents, especially in big cities of China. To solve the problem, the government has been increasing funds for improving the traffic infrastructure, enforcing traffic laws, and educating drivers about traffic regulations. In addition, research institutes have launched R&D projects in driver assistance and safety warning systems. In particular, in 1999, the Chinese Academy of Sciences' Intelligent Control and Systems Engineering Center started the Intelligent Vehicle Platforms project. The project aims to promote the use of intelligent technology for safe, efficient, and smart vehicles and to prototype vehicular electronic and sensory products and systems for the Chinese automotive industry. One of the project's key objectives is to develop a vehicular application-specific operating system (vASOS). The National Science Foundation of China and the CAS Knowledge Innovation Program support the project. Supported by the Vehicular Embedded Computing Platform project, the Xi'an Jiaotong University (XJTU-"Jiao Tong" means transportation in Chinese) Institute of Artificial Intelligence and Robotics and the CAS have collaborated to develop intelligent driver-assistance and safety warning systems for passenger vehicles, particularly GPS-and vision-based systems.