Arrow Research search

Author name cluster

Di Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

105 papers
2 author rows

Possible papers

105

AAAI Conference 2026 Conference Paper

A Hybrid Space Model for Misaligned Multi-modality Image Fusion

  • Yi Xiao
  • Jia Wang
  • Zhu Liu
  • Di Wang
  • Jinyuan Liu
  • Risheng Liu

Infrared and visible image fusion aims to integrate complementary information, such as thermal saliency from infrared imagery and fine-grained texture details from visible imagery. However, real-world multi-modal misalignment and geometric deformation often introduce severe artifacts. Most existing methods focus on feature extraction within Euclidean space, thereby neglecting the inherent hierarchical structures embedded in multimodal representations. While Euclidean space excels at preserving local structural details and supporting efficient computation, hyperbolic space is naturally suited for modeling hierarchical relationships due to its geometric properties. Building upon these observations, this paper proposes a unified framework that jointly optimizes image registration and fusion through a dual-space architecture. This architecture synergistically combines the local fidelity of Euclidean geometry with the hierarchical modeling capability of hyperbolic geometry to enhance multimodal representation learning. Specifically, this paper introduces Hyperbolic Coupled Contrastive Learning Optimization (HCCLO), which aligns and optimizes the hierarchical structures of infrared and visible embeddings in hyperbolic space. Moreover, this paper designs a task-adaptive dual-space features fusion mechanism, which dynamically balances and fuses Euclidean local features with hyperbolic hierarchical representations, thereby improving adaptability for downstream tasks. Extensive experiments on misaligned multimodal datasets demonstrate that our method achieves state-of-the-art performance, while effectively capturing both spatial dependencies and hierarchical semantics.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Anatomical Region-Guided Contrastive Decoding: A Plug-and-Play Strategy for Mitigating Hallucinations in Medical VLMs

  • Xiao Liang
  • Chenxi Liu
  • Zhi Ma
  • Di Wang
  • Bin Jing
  • Quan Wang
  • Yuanyuan Shi

Medical Vision-Language Models (MedVLMs) show immense promise in clinical applicability. However, their reliability is hindered by hallucinations, where models often fail to derive answers from visual evidence, instead relying on learned textual priors. Existing mitigation strategies for MedVLMs have distinct limitations: training-based methods rely on costly expert annotations, limiting scalability, while training-free interventions like contrastive decoding, though data-efficient, apply a global, untargeted correction whose effects in complex real-world clinical settings can be unreliable. To address these challenges, we introduce Anatomical Region-Guided Contrastive Decoding (ARCD), a plug-and-play strategy that mitigates hallucinations by providing targeted, region-specific guidance. Our module leverages an anatomical mask to direct a three-tiered contrastive decoding process. By dynamically re-weighting at the token, attention, and logits levels, it verifiably steers the model's focus onto specified regions, reinforcing anatomical understanding and suppressing factually incorrect outputs. Extensive experiments across diverse datasets, including chest X-ray, CT, brain MRI, and ocular ultrasound, demonstrate our method's effectiveness in improving regional understanding, reducing hallucinations, and enhancing overall diagnostic accuracy.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

Differentially Private Non-convex Learning: From Generalized Linear Models to Multi-layer Neural Networks

  • Hanpu Shen
  • Cheng-Long Wang
  • Zihang Xiang
  • Yiming Ying
  • Di Wang

This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node. In the first part, we examine cases with no hidden nodes, i. e. , Generalized Linear Models (GLMs). We investigate the well-specific model where the random noise possesses a zero mean, and the link (activation) function is both bounded and Lipschitz. We propose several algorithms and our analysis demonstrates the feasibility of achieving an excess population risk that remains invariant to the data dimension. We then delve into the scenario involving the ReLU link function, and our findings mirror those of the bounded link function. Moreover, we extend our ideas to two-layer neural networks with sigmoid or ReLU activation functions in the well-specified model. We conclude this section by contrasting well-specified and misspecified models, using ReLU regression as a representative example. In the second part, we study the theoretical guarantees of DP-SGD in Abadi et al. (2016) for multi-layer neural networks. By utilizing recent advances in Neural Tangent Kernel theory, we provide the first excess population risk when both the sample size and the width of the network are sufficiently large. Additionally, we discuss the role of some parameters in DP-SGD regarding their utility, both theoretically and empirically. 1

PDF

AAAI Conference 2026 Conference Paper

Efficient Few-Step Solution Generation via Discrete Flow Matching for Combinatorial Optimization

  • Yuanshu Li
  • Di Wang
  • Wei Du
  • Xuan Wu
  • Peng Zhao
  • Yubin Xiao
  • You Zhou

Combinatorial optimization problems (COPs) are fundamental to many real-world applications where efficiently producing high-quality solutions is critical. Recent advances in diffusion-based non-autoregressive models have reformulated solving COPs as a generative process, achieving promising results. However, almost all of these methods still suffer from accumulated errors and high inference costs due to the multi-step stochastic denoising process. To address these issues, we propose EFLOCO, an efficient discrete flow matching method for solving COPs, learning structured and deterministic solution trajectories. EFLOCO replaces noise-driven updates with smooth and guided transitions, thereby improves inference stability and quality. Furthermore, we introduce an adaptive time-step scheduler that makes more efforts in critical transition regions, yielding strong performance under few-step constraints. Experiments on standard Traveling Salesman Problems (TSPs) and Asymmetric TSPs (ATSPs) show that our method consistently outperforms both learning-based and heuristic baselines in terms of solution quality and inference speed.

PDF Details DOI

AAAI Conference 2026 Conference Paper

HFR-MKGC: Hierarchical Fusion Reasoning with MLLMs for Multi-modal Knowledge Graph Completion

  • Di Wang
  • Junping Du
  • Zhe Xue
  • Meiyu Liang
  • Guanhua Ye
  • Yingxia Shao
  • Haisheng Li

Multi-modal knowledge graph completion (MMKGC) aims to infer missing entities of triples by leveraging heterogeneous information in knowledge graph (KG). However, existing approaches often struggle with inconsistent modality alignment, limited reasoning depth, and insufficient negative sample quality. In this work, we propose HFR-MKGC, a novel framework that integrates hierarchical modal fusion and Multimodal Large Language Model (MLLM) reasoning for robust and expressive MMKGC. Specifically, we introduce a relation-guided hierarchical modal fusion module, which conducts fine-grained intra-visual fusion and relation-guided cross-modal integration to yield rich entity representations. HFR-MKGC employs a fine-tuned MLLM to perform instruction-based triple reasoning, producing candidate entities for completion. Then, it constructs hard negative samples through textual perturbation by MLLM and visual feature augmentation with rotation and noise. HFR-MKGC optimizes the model via adversarial training. Extensive experiments on three MMKGC benchmarks demonstrate that our method outperforms state-of-the-art methods, validating its effectiveness in MMKGC.

PDF Details DOI

AAAI Conference 2026 Conference Paper

RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images

  • Ke Li
  • Di Wang
  • Ting Wang
  • Fuyu Dong
  • Yiming Zhang
  • Luyao Zhang
  • Xiangyu Wang
  • Shaofeng Li

Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing images based on free-form natural language expressions. Existing approaches are typically constrained to closed-set vocabularies, limiting their applicability in open-world scenarios. While recent attempts to leverage generic foundation models for open-vocabulary RSVG, they overly rely on expensive high-quality datasets and time-consuming fine-tuning. To address these limitations, we propose RSVG-ZeroOV, a training-free framework that aims to explore the potential of frozen generic foundation models for zero-shot open-vocabulary RSVG. Specifically, RSVG-ZeroOV comprises three key stages: (i) Overview: We utilize a vision-language model (VLM) to obtain cross-attention maps that capture semantic correlations between text queries and visual regions. (ii) Focus: By leveraging the fine-grained modeling priors of a diffusion model (DM), we fill in gaps in structural and shape information of objects, which are often overlooked by VLM. (iii) Evolve: A simple yet effective attention evolution module is introduced to suppress irrelevant activations, yielding purified segmentation masks over the referred objects. Without cumbersome task-specific training, RSVG-ZeroOV offers an efficient and scalable solution. Extensive experiments demonstrate that the proposed framework consistently outperforms existing weakly-supervised and zero-shot methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

S5: Scalable Semi-Supervised Semantic Segmentation in Remote Sensing

  • Liang Lv
  • Di Wang
  • Jing Zhang
  • Lefei Zhang

Semi-supervised semantic segmentation (S4) has advanced remote sensing (RS) analysis by leveraging unlabeled data through pseudo-labeling and consistency learning. However, existing S4 studies often rely on small-scale datasets and models, limiting their practical applicability. To address this, we propose S5, the first scalable framework for semi-supervised semantic segmentation in RS, which unlocks the potential of vast unlabeled Earth observation data typically underutilized due to costly pixel-level annotations. Built upon existing large-scale RS datasets, S5 introduces a data selection strategy that integrates entropy-based filtering and diversity expansion, resulting in the RS4P-1M dataset. Using this dataset, we systematically scale up S4 into a new pretraining paradigm, S4 pre-training (S4P), to pretrain RS foundation models (RSFMs) of varying sizes on this extensive corpus, significantly boosting their performance on land cover segmentation and object detection tasks. Furthermore, during fine-tuning, we incorporate a Mixture-of-Experts (MoE)-based multi-dataset fine-tuning approach, which enables efficient adaptation to multiple RS benchmarks with fewer parameters. This approach improves the generalization and versatility of RSFMs across diverse RS benchmarks. The resulting RSFMs achieve state-of-the-art performance across all benchmarks, underscoring the viability of scaling semi-supervised learning for RS applications.

PDF Details DOI

AAAI Conference 2026 Conference Paper

SAGE: Spuriousness-Aware Guided Prompt Exploration for Mitigating Multimodal Bias

  • Wenqian Ye
  • Di Wang
  • Guangtao Zheng
  • Bohan Liu
  • Aidong Zhang

Large vision-language models such as CLIP have shown strong zero-shot classification performance by aligning images and text in a shared embedding space. However, CLIP models often develop multimodal spurious biases, the undesirable tendency to rely on spurious features. For example, CLIP may infer object types in images based on frequently co-occurring backgrounds rather than the object's core features. This bias significantly impairs the robustness of pre-trained CLIP models on out-of-distribution data, where such cross-modal associations no longer hold. Existing methods for mitigating multimodal spurious bias typically require fine-tuning on downstream data or prior knowledge of the bias, which undermines the out-of-the-box usability of CLIP. In this paper, we first theoretically analyze the impact of multimodal spurious bias in zero-shot classification. Based on this insight, we propose Spuriousness-Aware Guided Exploration (SAGE), a simple and effective method that mitigates spurious bias via guided prompt selection. SAGE requires no training, fine-tuning, or external annotations. It explores on a space of prompt templates and selects the prompts that induces the largest semantic separation between classes, thereby improving worst-group robustness. Extensive experiments on four real-world benchmark datasets and five popular backbone models demonstrate that SAGE consistently improves zero-shot performance and generalization, outperforming previous zero-shot approaches without any external knowledge or model updates.

PDF Details DOI

AAAI Conference 2026 Conference Paper

SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency

  • Zeqing Wang
  • Kangye Ji
  • Di Wang
  • Haibin Zhang
  • Fei Cheng

Segment Anything Model (SAM) struggles in open-world scenarios with diverse domains. In such settings, naive fine-tuning with a well-designed learning module is inadequate and often causes catastrophic forgetting issue when learning incrementally. To address this issue, we propose a novel continual learning (CL) method for SAM, termed SAMCL. Rather than relying on a fixed learning module, our method decomposes incremental knowledge into separate modules and trains a selector to choose the appropriate one during inference. However, this intuitive design introduces two key challenges: ensuring effective module learning and selection, and managing storage as tasks accumulate. To tackle these, we introduce two components: AugModule and Module Selector. AugModule reduces the storage of the popular LoRA learning module by sharing parameters across layers while maintaining accuracy. It also employs heatmaps—generated from point prompts—to further enhance domain adaptation with minimal additional cost. Module Selector leverages the observation that SAM’s embeddings can effectively distinguish domains, enabling high selection accuracy by training on low-consumed embeddings instead of raw images. Experiments show that SAMCL outperforms state-of-the-art methods, achieving only 0.19% forgetting and at least 2.5% gain on unseen domains. Each AugModule requires just 0.233 MB, reducing storage by at least 24.3% over other fine-tuning approaches. The buffer storage for Module Selector is further reduced by up to 256x.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

The Web Tool Trap: Understanding and Mitigating Over-Reliance in Browsing Agents

  • Jiawei Guo
  • Hongjie Nie
  • Qianbo Zang
  • Shu Yang
  • Shuodi Liu
  • Yiwei Ru
  • Liuyu Xiang
  • Di Wang

Large Language Model (LLM) agents that browse the web are increasingly important, but their effectiveness is hindered by imperfect integration of internal knowledge and external tools. We introduce BrowseBench and present the first systematic investigation into over-reliance patterns of browsing agents. Through controlled experiments, we identify three distinct failure modes: (1) Excessive Conservatism—unnecessary tool invocations for known information; (2) Over-trust in Web Sources—uncritical acceptance of retrieved content; and (3) Planning Deficiency—lack of query decomposition strategies. To address these, we propose three mitigation strategies: Direct Preference Optimization (DPO), Attention Refinement (AR), and Hierarchical Query Decomposition (HQD). Experiments demonstrate that our interventions significantly reduce over-reliance and enhance performance.

PDF

AAAI Conference 2026 Conference Paper

TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model

  • Yixing Li
  • Ruobing Xie
  • Zhen Yang
  • Xingwu Sun
  • Shuaipeng Li
  • Weidong Han
  • Zhanhui Kang
  • Di Wang

Transformers are the cornerstone of modern large language models, but their quadratic computational complexity limits efficiency in long-sequence processing. Recent advancements in Mamba, a state space model (SSM) with linear complexity, offer promising efficiency gains but suffer from unstable contextual learning and multitask generalization. Some works conduct layer-level hybrid structures that combine Transformer and Mamba layers, aiming to make full use of both advantages. This paper proposes TransMamba, a novel sequence-level hybrid framework that unifies Transformer and Mamba through shared parameter matrices (QKV and CBx), and thus could dynamically switch between attention and SSM mechanisms at different token lengths and layers. We design the Memory Converter to bridge Transformer and Mamba by converting attention outputs into SSM-compatible states, ensuring seamless information flow at TransPoints where the transformation happens. The TransPoint scheduling is also thoroughly explored for balancing effectiveness and efficiency. We conducted extensive experiments demonstrating that TransMamba achieves superior training efficiency and performance compared to single and hybrid baselines, and validated the deeper consistency between Transformer and Mamba paradigms at sequence level, offering a scalable solution for next-generation language modeling.

PDF Details DOI

AAAI Conference 2026 Conference Paper

When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models

  • Keyu Wang
  • Jin Li
  • Shu Yang
  • Zhuoran Zhang
  • Di Wang

Large Language Models (LLMs) often exhibit sycophantic behavior, agreeing with user-stated opinions even when those contradict factual knowledge. While prior work has documented this tendency, the internal mechanisms that enable such behavior remain poorly understood. In this paper, we provide a mechanistic account of how sycophancy arises within LLMs. We first systematically study how user opinions induce sycophancy across different model families. We find that simple opinion statements reliably induce sycophancy, whereas user expertise framing has a negligible impact. Through logit-lens analysis and causal activation patching, we identify a two-stage emergence of sycophancy: (1) a late-layer output preference shift and (2) deeper representational divergence. We also verify that user authority fails to influence behavior because models do not encode it internally. In addition, we examine how grammatical perspective affects sycophantic behavior, finding that first-person prompts (“I believe...”) consistently induce higher sycophancy rates than third-person framings (“They believe...”) by creating stronger representational perturbations in deeper layers. These findings highlight that sycophancy is not a surface-level artifact but emerges from a structural override of learned knowledge in deeper layers, with implications for alignment and truthful AI systems.

PDF Details DOI

ICLR Conference 2025 Conference Paper

A Benchmark for Semantic Sensitive Information in LLMs Outputs

  • Qingjie Zhang
  • Han Qiu 0001
  • Di Wang
  • Yiming Li 0004
  • Tianwei Zhang 0004
  • Wenyu Zhu
  • Haiqin Weng
  • Liu Yan

Large language models (LLMs) can output sensitive information, which has emerged as a novel safety concern. Previous works focus on structured sensitive information (e.g. personal identifiable information). However, we notice that sensitive information can also be at semantic level, i.e. semantic sensitive information (SemSI). Particularly, *simple natural questions* can let state-of-the-art (SOTA) LLMs output SemSI. %which is hard to be detected compared with structured ones. Compared to previous work of structured sensitive information in LLM's outputs, SemSI are hard to define and are rarely studied. Therefore, we propose a novel and large-scale investigation on the existence of SemSI in SOTA LLMs induced by simple natural questions. First, we construct a comprehensive and labeled dataset of semantic sensitive information, SemSI-Set, by including three typical categories of SemSI. Then, we propose a large-scale benchmark, SemSI-Bench, to systematically evaluate semantic sensitive information in 25 SOTA LLMs. Our finding reveals that SemSI widely exists in SOTA LLMs' outputs by querying with simple natural questions. We open-source our project at https://semsi-project.github.io/.

Details

ICLR Conference 2025 Conference Paper

A Solvable Attention for Neural Scaling Laws

  • Bochen Lyu
  • Di Wang
  • Zhanxing Zhu

Transformers and many other deep learning models are empirically shown to predictably enhance their performance as a power law in training time, model size, or the number of training data points, which is termed as the neural scaling law. This paper studies this intriguing phenomenon particularly for the transformer architecture in theoretical setups. Specifically, we propose a framework for linear self-attention, the underpinning block of transformer without softmax, to learn in an in-context manner, where the corresponding learning dynamics is modeled as a non-linear ordinary differential equation (ODE) system. Furthermore, we establish a procedure to derive a tractable approximate solution for this ODE system by reformulating it as a *Riccati equation*, which allows us to precisely characterize neural scaling laws for linear self-attention with training time, model size, data size, and the optimal compute. In addition, we reveal that the linear self-attention shares similar neural scaling laws with several other architectures when the context sequence length of the in-context learning is fixed, otherwise it would exhibit a different scaling law of training time.

Details

IJCAI Conference 2025 Conference Paper

ABNet: Mitigating Sample Imbalance in Anomaly Detection Within Dynamic Graphs

  • Yifan Hong
  • Muhammad Asif Ali
  • Huan Wang
  • Junyang Chen
  • Di Wang

In dynamic graphs, detecting anomalous nodes faces challenges due to sample imbalance, stemming from the scarcity of anomalous samples and feature representation bias. Existing methods often use unsupervised or semi-supervised learning to extract anomalous samples from unlabeled data, but struggle to obtain enough anomalous instances due to their low occurrence. Moreover, GNN-based approaches often prioritize normal samples, neglecting rare anomalies. To address these issues, we propose the Anomaly Balance Network (ABNet), designed to alleviate sample imbalance and enhance anomaly detection. ABNet includes three key components: a feature extractor that compares node features across time points to avoid bias, an anomaly augmenter that amplifies anomaly details and generates diverse anomalous samples, and an anomaly detector using meta-learning to adapt to graph evolution. Experimental results show that ABNet outperforms existing methods on three real-world datasets, effectively addressing sample imbalance.

PDF Details DOI

JMLR Journal 2025 Journal Article

Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed Learning Scheme for Data Silos

  • Shao-Bo Lin
  • Xiaotong Liu
  • Di Wang
  • Hai Zhang
  • Ding-Xuan Zhou

Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose. Distributed learning based on divide-and-conquer provides a promising way to settle the data silos, but it suffers from several challenges, including autonomy, privacy guarantees, and the necessity of collaborations. This paper focuses on developing an adaptive distributed kernel ridge regression (AdaDKRR) by taking autonomy in parameter selection, privacy in communicating non-sensitive information, and the necessity of collaborations for performance improvement into account. We provide both solid theoretical verifications and comprehensive experiments for AdaDKRR to demonstrate its feasibility and effectiveness. Theoretically, we prove that under some mild conditions, AdaDKRR performs similarly to running the optimal learning algorithms on the whole data, verifying the necessity of collaborations and showing that no other distributed learning scheme can essentially beat AdaDKRR under the same conditions. Numerically, we test AdaDKRR on both toy simulations and two real-world applications to show that AdaDKRR is superior to other existing distributed learning schemes. All these results show that AdaDKRR is a feasible scheme to overcome data silos, which are highly desired in numerous application regions such as intelligent decision-making, pricing forecasting, and performance prediction for products. [abs] [ pdf ][ bib ] &copy JMLR 2025. ( edit, beta )

PDF Details

TMLR Journal 2025 Journal Article

Beyond ordinary Lipschitz constraints: Differentially Private optimization with TNC

  • Difei Xu
  • Meng Ding
  • Zihang Xiang
  • Jinhui Xu
  • Di Wang

We study Stochastic Convex Optimization in Differential Privacy model (DP-SCO). Unlike previous studies, here we assume the population risk function satisfies the Tsybakov Noise Condition (TNC) with some parameter $\theta>1$, where the Lipschitz constant of the loss could be extremely large or even unbounded, but the $\ell_2$-norm gradient of the loss has bounded $k$-th moment with $k\geq 2$. For the Lipschitz case with $\theta\geq 2$, we first propose an $(\epsilon, \delta)$-DP algorithms whose utility bound is $\tilde{O}\left(\left(\tilde{r}_{2k}(\frac{1}{\sqrt{n}}+(\frac{\sqrt{d}}{n\epsilon}))^\frac{k-1}{k}\right)^\frac{\theta}{\theta-1}\right)$ in high probability, where $n$ is the sample size, $d$ is the model dimension, and $\tilde{r}_{2k}$ is a term that only depends on the $2k$-th moment of the gradient. It is notable that such an upper bound is independent of the Lipschitz constant. We then extend to the case where $\theta\geq \bar{\theta}> 1$ for some known constant $\bar{\theta}$. Moreover, when the privacy budget $\epsilon$ is small enough, we show an upper bound of $\tilde{O}\left(\left(\tilde{r}_{k}(\frac{1}{\sqrt{n}}+(\frac{\sqrt{d}}{n\epsilon}))^\frac{k-1}{k}\right)^\frac{\theta}{\theta-1}\right)$ even if the loss function is not Lipschitz. For the lower bound, we show that for any $\theta\geq 2$, the private minimax rate for $\rho$-zero Concentrated Differential Privacy is lower bounded by $\Omega\left(\left(\tilde{r}_{k}(\frac{1}{\sqrt{n}}+(\frac{\sqrt{d}}{n\sqrt{\rho}}))^\frac{k-1}{k}\right)^\frac{\theta}{\theta-1}\right)$.

PDF Details

AAAI Conference 2025 Conference Paper

CognitionCapturer: Decoding Visual Stimuli from Human EEG Signal with Multimodal Information

  • Kaifan Zhang
  • Lihuo He
  • Xin Jiang
  • Wen Lu
  • Di Wang
  • Xinbo Gao

Electroencephalogram (EEG) signals have attracted significant attention from researchers due to their non-invasive nature and high temporal sensitivity in decoding visual stimuli. However, most recent studies have focused solely on the relationship between EEG and image data pairs, neglecting the valuable "beyond-image-modality" information embedded in EEG signals. This results in the loss of critical multimodal information in EEG. To address the limitation, this paper proposes a unified framework that fully leverages multimodal data to represent EEG signals, named CognitionCapturer. Specifically, CognitionCapturer trains modality expert encoders for each modality to extract cross-modal information from the EEG modality. Then, it introduces a diffusion prior to map the EEG embedding space to the CLIP embedding space, followed by using a pretrained generative model, the proposed framework can reconstruct visual stimuli with high semantic and structural fidelity. Notably, the framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities. Through extensive experiments, we demonstrate that CognitionCapturer outperforms state-of-the-art methods both qualitatively and quantitatively.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

DGL: Dynamic Global-Local Information Aggregation for Scalable VRP Generalization with Self-Improvement Learning

  • Yubin Xiao
  • Yuesong Wu
  • Rui Cao
  • Di Wang
  • Zhiguang Cao
  • Xuan Wu
  • Peng Zhao
  • Yuanshu Li

The Vehicle Routing Problem (VRP) is a critical combinatorial optimization problem with wide-reaching real-world applications, particularly in logistics, transportation. While neural network-based VRP solvers have shown impressive results on test instances similar to training data, their performance often degrades when faced with varying scales and unseen distributions, limiting their practical applicability. To overcome these limitations, we introduce DGL (Dynamic Global-Local Information Aggregation), a novel model that combines global and local information to effectively solve VRPs. DGL dynamically adjusts local node selections within a localized range, capturing local invariance across problems of different scales and distributions, thereby enhancing generalization. At the same time, DGL integrates global context into the decision-making process, providing richer information for more informed decisions. Additionally, we propose a replacement-based self-improvement learning framework that leverages data augmentation and random replacement techniques, further enhancing DGL's robustness. Extensive experiments on synthetic datasets, benchmark datasets, and real-world country map instances demonstrate that DGL achieves state-of-the-art performance, particularly in generalizing to large-scale VRPs and real-world scenarios. These results showcase DGL's effectiveness in solving complex, realistic optimization challenges and highlight its potential for practical applications.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration

  • Hebaixu Wang
  • Jing Zhang
  • Haonan Guo
  • Di Wang
  • Jiayi Ma
  • Bo Du

Diffusion models have achieved remarkable progress in universal image restoration. However, existing methods perform naive inference in the reverse process, which leads to cumulative errors under limited sampling steps and large step intervals. Moreover, they struggle to balance the commonality of degradation representations with restoration quality, often depending on complex compensation mechanisms that enhance fidelity at the expense of efficiency. To address these challenges, we introduce \textbf{DGSolver}, a diffusion generalist solver with universal posterior sampling. We first derive the exact ordinary differential equations for generalist diffusion models to unify degradation representations and design tailored high-order solvers with a queue-based accelerated sampling strategy to improve both accuracy and efficiency. We then integrate universal posterior sampling to better approximate manifold-constrained gradients, yielding a more accurate noise estimation and correcting errors in inverse inference. Extensive experiments demonstrate that DGSolver outperforms state-of-the-art methods in restoration accuracy, stability, and scalability, both qualitatively and quantitatively. Code and models are publicly available at https: //github. com/MiliLab/DGSolver.

PDF Details

NeurIPS Conference 2025 Conference Paper

EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification

  • Lin Zhang
  • Wenshuo Dong
  • Zhuoran Zhang
  • Shu Yang
  • Lijie Hu
  • Ninghao Liu
  • Pan Zhou
  • Di Wang

Understanding the internal mechanisms of transformer-based language models remains challenging. Mechanistic interpretability based on circuit discovery aims to reverse engineer neural networks by analyzing their internal processes at the level of computational subgraphs. In this paper, we revisit existing gradient-based circuit identification methods and find that their performance is either affected by the zero-gradient problem or saturation effects, where edge attribution scores become insensitive to input changes, resulting in noisy and unreliable attribution evaluations for circuit components. To address the saturation effect, we propose Edge Attribution Patching with GradPath (EAP-GP), EAP-GP introduces an integration path, starting from the input and adaptively following the direction of the difference between the gradients of corrupted and clean inputs to avoid the saturated region. This approach enhances attribution reliability and improves the faithfulness of circuit identification. We evaluate EAP-GP on 6 datasets using GPT-2 Small, GPT-2 Medium, and GPT-2 XL. Experimental results demonstrate that EAP-GP outperforms existing methods in circuit faithfulness, achieving improvements up to 17. 7\%. Comparisons with manually annotated ground-truth circuits demonstrate that EAP-GP achieves precision and recall comparable to or better than previous approaches, highlighting its effectiveness in identifying accurate circuits.

PDF Details

AAAI Conference 2025 Conference Paper

Fair Text-to-Image Diffusion via Fair Mapping

  • Jia Li
  • Lijie Hu
  • Jingfeng Zhang
  • Tianhang Zheng
  • Hua Zhang
  • Di Wang

In this paper, we address the limitations of existing text-to-image diffusion models in generating demographically fair results when given human-related descriptions. These models often struggle to disentangle the target language context from sociocultural biases, resulting in biased image generation. To overcome this challenge, we propose Fair Mapping, a flexible, model-agnostic, and lightweight approach that modifies a pre-trained text-to-image diffusion model by controlling the prompt to achieve fair image generation. One key advantage of our approach is its high efficiency. It only requires updating an additional linear network with few parameters at a low computational cost. By developing a linear network that maps conditioning embeddings into a debiased space, we enable the generation of relatively balanced demographic results based on the specified text condition. With comprehensive experiments on face image generation, we show that our method significantly improves image generation fairness with almost the same image quality compared to conventional diffusion models when prompted with descriptions related to humans. By effectively addressing the issue of implicit language bias, our method produces more fair and diverse image outputs.

PDF Details DOI

TMLR Journal 2025 Journal Article

Faithful Interpretation for Graph Neural Networks

  • Lijie Hu
  • Tianhao Huang
  • Lu Yu
  • Wanyu Lin
  • Tianhang Zheng
  • Di Wang

Currently, attention mechanisms have garnered increasing attention in Graph Neural Networks (GNNs), such as Graph Attention Networks (GATs) and Graph Transformers (GTs). This is due to not only the commendable boost in performance they offer but also their capacity to provide a more lucid rationale for model behaviors, which are often viewed as inscrutable. However, Attention-based GNNs have demonstrated instability in interpretability when subjected to various sources of perturbations during both training and testing phases, including factors like additional edges or nodes. In this paper, we propose a solution to this problem by introducing a novel notion called Faithful Graph Attention-based Interpretation (FGAI). In particular, FGAI has four crucial properties in terms of stability and sensitivity to interpretation and the final output distribution. Built upon this notion, we propose an efficient methodology for obtaining FGAI, which can be viewed as an ad hoc modification to the canonical Attention-based GNNs. To validate our proposed solution, we introduce two novel metrics tailored for graph interpretation assessment. Experimental results demonstrate that FGAI exhibits superior stability and preserves the interpretability of attention under various forms of perturbations and randomness, which makes FGAI a more faithful and reliable explanation tool.

PDF Details

AAAI Conference 2025 Conference Paper

FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection

  • Ke Li
  • Di Wang
  • Zhangyuan Hu
  • Shaofeng Li
  • Weiping Ni
  • Lin Zhao
  • Quan Wang

Infrared-visible object detection (IVOD) seeks to harness the complementary information in infrared and visible images, thereby enhancing the performance of detectors in complex environments. However, existing methods often neglect the frequency characteristics of complementary information, such as the abundant high-frequency details in visible images and the valuable low-frequency thermal information in infrared images, thus constraining detection performance. To solve this problem, we introduce a novel Frequency-Driven Feature Decomposition Network for IVOD, called FD2-Net, which effectively captures the unique frequency representations of complementary information across multimodal visual spaces. Specifically, we propose a feature decomposition encoder, wherein the high-frequency unit (HFU) utilizes discrete cosine transform to capture representative high-frequency features, while the low-frequency unit (LFU) employs dynamic receptive fields to model the multi-scale context of diverse objects. Next, we adopt a parameter-free complementary strengths strategy to enhance multimodal features through seamless inter-frequency recoupling. Furthermore, we innovatively design a multimodal reconstruction mechanism that recovers image details lost during feature extraction, further leveraging the complementary information from infrared and visible images to enhance overall representational capacity. Extensive experiments demonstrate that FD2-Net outperforms state-of-the-art (SoTA) models across various IVOD benchmarks, i.e. LLVIP (96.2% mAP), FLIR (82.9% mAP), and M3FD (83.5% mAP).

PDF Details DOI

EAAI Journal 2025 Journal Article

Few-shot machine reading comprehension for bridge inspection via domain-specific and task-aware pre-tuning approach

  • Ren Li
  • Luyi Zhang
  • Qiao Xiao
  • Jianxi Yang
  • Yu Chen
  • Shixin Jiang
  • Di Wang

With the wide application of information technologies in the field of bridge engineering, many electronic bridge inspection reports have been generated. However, due to insufficient research on machine reading comprehension (MRC) in this field, a lot of bridge inspection information, e. g. , structural basic data, inspected defects, and maintenance suggestions, has not been fully used. Especially, it is time-consuming and labor-intensive to pre-train a domain-specific language model from scratch or annotate large-scale question answering corpora, which also brings challenges to the MRC research in this field. To tackle the problems, this paper proposes a novel few-shot MRC approach for bridge inspection based on the idea of data augmentation. The proposed model uses a pre-trained model as backbone, along with introducing a pre-tuning stage to bridge the gaps between general-purpose pre-training and domain-specific MRC tasks. In order to reduce the workload of manual annotation, we present a novel pre-tuning data generation algorithm which is based on the domain-specific question classification and answer prediction neural models. After pre-tuning and fine-tuning, the proposed model achieves efficient bridge inspection MRC. The experimental results show that the proposed model outperforms the mainstream fine-tuning-based approaches and few-shot MRC baseline models in various settings. With 1024 fine-tuning samples, the F1 value and Exact Match (EM) value are 86. 42%, 74. 65%, respectively. Our research work can serve as a foundation for the construction of automatic question answering systems for intelligent bridge management and maintenance.

Details DOI

NeurIPS Conference 2025 Conference Paper

GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution

  • Fengxiang Wang
  • Mingshuo Chen
  • Yueying Li
  • Di Wang
  • Haotian Wang
  • Zonghao Guo
  • Zefan Wang
  • Shan Boqi

Ultra-high-resolution (UHR) remote sensing (RS) imagery offers valuable data for Earth observation but pose challenges for existing multimodal foundation models due to two key bottlenecks: (1) limited availability of UHR training data, and (2) token explosion caused by the large image size. To address data scarcity, we introduce **SuperRS-VQA** (avg. 8, 376$\times$8, 376) and **HighRS-VQA** (avg. 2, 000$\times$1, 912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world dialogue tasks. To mitigate token explosion, our pilot studies reveal significant redundancy in RS images: crucial information is concentrated in a small subset of object-centric tokens, while pruning background tokens (e. g. , ocean or forest) can even improve performance. Motivated by these findings, we propose two strategies: *Background Token Pruning* and *Anchored Token Selection*, to reduce the memory footprint while preserving key semantics. Integrating these techniques, we introduce **GeoLLaVA-8K**, the first RS-focused multimodal large language model capable of handling inputs up to 8K$\times$8K resolution, built on the LLaVA framework. Trained on SuperRS-VQA and HighRS-VQA, GeoLLaVA-8K sets a new state-of-the-art on the XLRS-Bench. Datasets and code were released at https: //github. com/MiliLab/GeoLLaVA-8K.

PDF Details

AAAI Conference 2025 Conference Paper

Improved Rates of Differentially Private Nonconvex-Strongly-Concave Minimax Optimization

  • Ruijia Zhang
  • Mingxi Lei
  • Meng Ding
  • Zihang Xiang
  • Jinhui Xu
  • Di Wang

In this paper, we study the problem of (finite sum) minimax optimization in the Differential Privacy (DP) model. Unlike most of the previous studies on the (strongly) convex-concave settings or loss functions satisfying the Polyak-Lojasiewicz condition, here we mainly focus on the nonconvex-strongly-concave one, which encapsulates many models in deep learning such as deep AUC maximization. Specifically, we first analyze a DP version of Stochastic Gradient Descent Ascent (SGDA) and show the utility bound in terms of the Euclidean norm of the gradient for the empirical risk function. We then propose a new method with less gradient noise variance and improve the upper bound to the best-known result for DP Empirical Risk Minimization with non-convex loss. We also discussed several lower bounds of private minimax optimization. Finally, experiments on AUC maximization, generative adversarial networks, and temporal difference learning with real-world data support our theoretical analysis.

PDF Details DOI

JBHI Journal 2025 Journal Article

LLM-Enhanced Multi-Teacher Knowledge Distillation for Modality-Incomplete Emotion Recognition in Daily Healthcare

  • Yuzhe Zhang
  • Huan Liu
  • Yang Xiao
  • Mohammed Amoon
  • Dalin Zhang
  • Di Wang
  • Shusen Yang
  • Chai Quek

The critical importance of monitoring and recognizing human emotional states in healthcare has led to a surge in proposals for EEG-based multimodal emotion recognition in recent years. However, practical challenges arise in acquiring EEG signals in daily healthcare settings due to stringent data acquisition conditions, resulting in the issue of incomplete modalities. Existing studies have turned to knowledge distillation as a means to mitigate this problem by transferring knowledge from multimodal networks to unimodal ones. However, these methods are constrained by the use of a single teacher model to transfer integrated feature extraction knowledge, particularly concerning spatial and temporal features in EEG data. To address this limitation, we propose a multi-teacher knowledge distillation framework enhanced with a Large Language Model (LLM), aimed at facilitating effective feature learning in the student network by transferring knowledge of extracting integrated features. Specifically, we employ an LLM as the teacher for extracting temporal features and a graph convolutional neural network for extracting spatial features. To further enhance knowledge distillation, we introduce causal masking and a confidence indicator into the LLM to facilitate the transfer of the most discriminative features. Extensive testing on the DEAP and MAHNOB-HCI datasets demonstrates that our model outperforms existing methods in the modality-incomplete scenario. This study underscores the potential application of large models in this field.

Details DOI

IJCAI Conference 2025 Conference Paper

Object-Level Backdoor Attacks in RGB-T Semantic Segmentation with Cross-Modality Trigger Optimization

  • Xianghao Jiao
  • Di Wang
  • Jiawei Liang
  • Jianjie Huang
  • Wei Wang
  • Xiaochun Cao

The escalating threat of backdoor risks in deep vision models is a pressing concern. Existing research on backdoor attacks is often confined to a single modality, neglecting the challenges posed by multi-modality scene perception. This work is a pioneer of backdoor attacks in RGB-Thermal (RGB-T) semantic segmentation. We overcome the critical limitation of current segmentation backdoor attacks that indiscriminately compromise all objects of a victim class, failing to provide fine-grained control for selectively targeting specific objects as required by adversaries. To address this, we introduce a novel Object-level Backdoor Attack pipeline, termed OBA. The OBA first employs a precise data poisoning (PDP) to lock a specific victim object. Specifically, the PDP embeds the trigger into the only victim object and modifies its label’s pixels at the corresponding positions, thus enabling object-level attacks. In addition, the domain gap between static single-modality triggers and multi-modality scenarios limits the PDP. We therefore introduce a Cross-Modality Trigger Generation (CMTG) method. Through style designs of triggers and cross-modality trigger co-optimization, the target domain semantics and multi-modality model perception patterns are encoded into triggers, achieving high effectiveness, stealth, and physical feasibility of triggers. Extensive experiments show that the proposed OBA enables precise manipulation of the designated object within the specific class.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Privacy-Preserving Low-Rank Adaptation Against Membership Inference Attacks for Latent Diffusion Models

  • Zihao Luo
  • Xilie Xu
  • Feng Liu
  • Yun Sing Koh
  • Di Wang
  • Jingfeng Zhang

Low-rank adaptation (LoRA) is an efficient strategy for adapting latent diffusion models (LDMs) on a private dataset to generate specific images by minimizing the adaptation loss. However, the LoRA-adapted LDMs are vulnerable to membership inference (MI) attacks that can judge whether a particular data point belongs to the private dataset, thus leading to the privacy leakage. To defend against MI attacks, we first propose a straightforward solution: Membership-Privacy-preserving LoRA (MP-LoRA). MP-LoRA is formulated as a min-max optimization problem where a proxy attack model is trained by maximizing its MI gain while the LDM is adapted by minimizing the sum of the adaptation loss and the MI gain of the proxy attack model. However, we empirically find that MP-LoRA has the issue of unstable optimization, and theoretically analyze that the potential reason is the unconstrained local smoothness, which impedes the privacy-preserving adaptation. To mitigate this issue, we further propose a Stable Membership-Privacy-preserving LoRA (SMP-LoRA) that adapts the LDM by minimizing the ratio of the adaptation loss to the MI gain. Besides, we theoretically prove that the local smoothness of SMP-LoRA can be constrained by the gradient norm, leading to improved convergence. Our experimental results corroborate that SMP-LoRA can indeed defend against MI attacks and generate high-quality images.

PDF Details DOI

TCS Journal 2025 Journal Article

Private least absolute deviations with heavy-tailed data

  • Di Wang
  • Jinhui Xu

We study the problem of Differentially Private Stochastic Convex Optimization (DPSCO) with heavy-tailed data. Specifically, we focus on the problem of Least Absolute Deviations, i. e. , ℓ 1 -norm linear regression, in the ϵ-DP model. While most previous work focuses on the case where the loss function is Lipschitz, in this paper we only need to assume the variates have bounded moments. Firstly, we study the case where the ℓ 2 norm of data has a bounded second-order moment. We propose an algorithm that is based on the exponential mechanism and show that it is possible to achieve an upper bound of O ˜ ( d n ϵ ) (with high probability). Next, we relax the assumption to bounded θ-th order moment with some θ ∈ ( 1, 2 ) and show that it is possible to achieve an upper bound of O ˜ ( ( d n ϵ ) θ − 1 θ ). Our algorithms can also be extended to more relaxed cases where only each coordinate of the data has bounded moments, and we can get an upper bound of O ˜ ( d n ϵ ) and O ˜ ( d ( n ϵ ) θ − 1 θ ) in the second and θ-th moment case respectively.

Details DOI

NeurIPS Conference 2025 Conference Paper

Private Training Large-scale Models with Efficient DP-SGD

  • Liangyu Wang
  • Junxiao Wang
  • Jie Ren
  • Zihang Xiang
  • David Keyes
  • Di Wang

As large language models (LLMs) increasingly underpin technological advancements, the privacy of their training data emerges as a critical concern. Differential Privacy (DP) serves as a rigorous mechanism to protect this data, yet its integration via Differentially Private Stochastic Gradient Descent (DP-SGD) introduces substantial challenges, primarily due to the complexities of per-sample gradient clipping. Current explicit methods, such as Opacus, necessitate extensive storage for per-sample gradients, significantly inflating memory requirements. Conversely, implicit methods like GhostClip reduce storage needs by recalculating gradients multiple times, which leads to inefficiencies due to redundant computations. This paper introduces FlashDP, an innovative cache-friendly per-layer DP-SGD that consolidates necessary operations into a single task, calculating gradients only once in a fused manner. This approach not only diminishes memory movement by up to 50\% but also cuts down redundant computations by 20\%, compared to previous methods. Consequently, FlashDP does not increase memory demands and achieves a 90\% throughput compared to the Non-DP method on a four-A100 system during the pre-training of the Llama-13B model, while maintaining parity with standard per-layer clipped DP-SGD in terms of accuracy. These advancements establish FlashDP as a pivotal development for efficient and privacy-preserving training of LLMs. FlashDP's code has been open-sourced in https: //github. com/kaustpradalab/flashdp.

PDF Details

AAAI Conference 2025 Conference Paper

ReMask-Animate: Refined Character Image Animation Using Mask-Guided Adapters

  • Xunzhi Xiang
  • Haiwei Xue
  • Zonghong Dai
  • Di Wang
  • Minglei Li
  • Ye Yue
  • Fei Ma
  • Weijiang Yu

Pose-controlled human video generation is of significant interest and finds extensive applications in areas such as automated advertising and content creation on social media platforms. While existing methods employing pose sequences and reference images for human image animation have exhibited notable performance, they tend to encounter issues such as specific region blurring, background sharpening, and decreased identity consistency. In this paper, we introduce ReMask-Animate, which utilizes masks as additional priors to guide the model's local visual attention to specific areas, thereby alleviating feature confusion between different regions of the image. Three distinct mask-guided adapters are designed for cross-condition regional fusion of hand and face pose features, mitigating feature confusion between the foreground and background, and enhancing the visual consistency of character identity. Moreover, these lightweight adapters introduce minimal computational overhead and can be seamlessly integrated into specific layers of the backbone architecture. Extensive experiments show that our method outperforms state-of-the-art methods on five metrics in public datasets. Additionally, qualitative evaluations highlight a significant improvement in the quality of generated videos, demonstrating our approach's superiority.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing

  • Fengxiang Wang
  • Yulin Wang
  • Mingshuo Chen
  • Haotian Wang
  • Hongzhen Wang
  • Haiyan Zhao
  • Yangang Sun
  • Shuo Wang

Recent advances in self-supervised learning for Vision Transformers (ViTs) have fueled breakthroughs in remote sensing (RS) foundation models. However, the quadratic complexity of self-attention poses a significant barrier to scalability, particularly for large models and high-resolution images. While the linear-complexity Mamba architecture offers a promising alternative, existing RS applications of Mamba remain limited to supervised tasks on small, domain-specific datasets. To address these challenges, we propose RoMA, a framework that enables scalable self-supervised pretraining of Mamba-based RS foundation models using large-scale, diverse, unlabeled data. RoMA enhances scalability for high-resolution images through a tailored auto-regressive learning strategy, incorporating two key innovations: 1) a rotation-aware pretraining mechanism combining adaptive cropping with angular embeddings to handle sparsely distributed objects with arbitrary orientations, and 2) multi-scale token prediction objectives that address the extreme variations in object scales inherent to RS imagery. Systematic empirical studies validate that Mamba adheres to RS data and parameter scaling laws, with performance scaling reliably as model and data size increase. Furthermore, experiments across scene classification, object detection, and semantic segmentation tasks demonstrate that RoMA-pretrained Mamba models consistently outperform ViT-based counterparts in both accuracy and computational efficiency. The source code and pretrained models have be released at https: //github. com/MiliLab/RoMA.

PDF Details

NeurIPS Conference 2025 Conference Paper

Second-Order Convergence in Private Stochastic Non-Convex Optimization

  • Youming Tao
  • Zuyuan Zhang
  • Dongxiao Yu
  • Xiuzhen Cheng
  • Falko Dressler
  • Di Wang

We investigate the problem of finding second-order stationary points (SOSP) in differentially private (DP) stochastic non-convex optimization. Existing methods suffer from two key limitations: \textbf{(i)} inaccurate convergence error rate due to overlooking gradient variance in the saddle point escape analysis, and \textbf{(ii)} dependence on auxiliary private model selection procedures for identifying DP-SOSP, which can significantly impair utility, particularly in distributed settings. To address these issues, we propose a generic perturbed stochastic gradient descent (PSGD) framework built upon Gaussian noise injection and general gradient oracles. A core innovation of our framework is using model drift distance to determine whether PSGD escapes saddle points, ensuring convergence to approximate local minima without relying on second-order information or additional DP-SOSP identification. By leveraging the adaptive DP-SPIDER estimator as a specific gradient oracle, we develop a new DP algorithm that rectifies the convergence error rates reported in prior work. We further extend this algorithm to distributed learning with heterogeneous data, providing the first formal guarantees for finding DP-SOSP in such settings. Our analysis also highlights the detrimental impacts of private selection procedures in distributed learning under high-dimensional models, underscoring the practical benefits of our design. Numerical experiments on real-world datasets validate the efficacy of our approach.

PDF Details

NeurIPS Conference 2025 Conference Paper

Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidence

  • Shaopeng Fu
  • Liang Ding
  • Jingfeng Zhang
  • Di Wang

Jailbreak attacks against large language models (LLMs) aim to induce harmful behaviors in LLMs through carefully crafted adversarial prompts. To mitigate attacks, one way is to perform adversarial training (AT)-based alignment, i. e. , training LLMs on some of the most adversarial prompts to help them learn how to behave safely under attacks. During AT, the length of adversarial prompts plays a critical role in the robustness of aligned LLMs. While long-length adversarial prompts during AT might lead to strong LLM robustness, their synthesis however is very resource-consuming, which may limit the application of LLM AT. This paper focuses on adversarial suffix jailbreak attacks and unveils that to defend against a jailbreak attack with an adversarial suffix of length $\Theta(M)$, it is enough to align LLMs on prompts with adversarial suffixes of length $\Theta(\sqrt{M})$. Theoretically, we analyze the adversarial in-context learning of linear transformers on linear regression tasks and prove a robust generalization bound for trained transformers. The bound depends on the term $\Theta(\sqrt{M_{\text{test}}}/M_{\text{train}})$, where $M_{\text{train}}$ and $M_{\text{test}}$ are the numbers of adversarially perturbed in-context samples during training and testing. Empirically, we conduct AT on popular open-source LLMs and evaluate their robustness against jailbreak attacks of different adversarial suffix lengths. Results confirm a positive correlation between the attack success rate and the ratio of the square root of the adversarial suffix length during jailbreaking to the length during AT. Our findings show that it is practical to defend against "long-length" jailbreak attacks via efficient "short-length" AT. The code is available at https: //github. com/fshp971/adv-icl.

PDF Details

NeurIPS Conference 2025 Conference Paper

TEMPO: Temporal Multi-scale Autoregressive Generation of Protein Conformational Ensembles

  • Yaoyao Xu
  • Di Wang
  • Zihan Zhou
  • Tianshu Yu
  • Mingchen Chen

Understanding the dynamic behavior of proteins is critical to elucidating their functional mechanisms, yet generating realistic, temporally coherent trajectories of protein ensembles remains a significant challenge. In this work, we introduce a novel hierarchical autoregressive framework for modeling protein dynamics that leverages the intrinsic multi-scale organization of molecular motions. Unlike existing methods that focus on generating static conformational ensembles or treat dynamic sampling as an independent process, our approach characterizes protein dynamics as a Markovian process. The framework employs a two-scale architecture: a low-resolution model captures slow, collective motions driving major conformational transitions, while a high-resolution model generates detailed local fluctuations conditioned on these large-scale movements. This hierarchical design ensures that the causal dependencies inherent in protein dynamics are preserved, enabling the generation of temporally coherent and physically realistic trajectories. By bridging high-level biophysical principles with state-of-the-art generative modeling, our approach provides an efficient framework for simulating protein dynamics that balances computational efficiency with physical accuracy.

PDF Details

IJCAI Conference 2025 Conference Paper

TextMEF: Text-guided Prompt Learning for Multi-exposure Image Fusion

  • Jinyuan Liu
  • Qianjun Huang
  • Guanyao Wu
  • Di Wang
  • Zhiying Jiang
  • Long Ma
  • Risheng Liu
  • Xin Fan

Multi-exposure image fusion~(MEF) aims to integrate a set of low dynamic range images, producing a single image with a higher dynamic range than either one. Despite significant advancements, current MEF approaches still struggle to handle extremely over- or under-exposed conditions, resulting in unsatisfactory visual effects such as hallucinated details and distorted color tones. With this regard, we propose TextMEF, a prompt-driven fusion method enhanced by prompt learning, for multi-exposure image fusion. Specifically, we learn a set of prompts based on text-image similarity among negative and positive samples (over-exposed, under-exposed images, and well-exposed ones). These learned prompts are seamlessly integrated into the loss function, providing high-level guidance for constraining non-uniform exposure regions. Furthermore, we develop a attention Mamba module effectively translates over-/under- exposed regional features into exposure invariant space and ensure them to build efficient long-range dependency to high dynamic range image. Extensive experimental results on three publicly available benchmarks demonstrate that our TextMEF significantly outperforms state-of-the-art approaches in both visual inspection and objective analysis.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Topology-Aware Learning of Tubular Manifolds via SE(3)-Equivariant Network on Ball B-Spline Curve

  • Jingxuan Wang
  • Zhongke Wu
  • Zhang Zeyao
  • Chunhao Zheng
  • Di Wang

Tubular-like system shape analysis is quite difficult in geometry and topology, while it is widely used in plants and organs analysis in practice. However, traditional discrete representations such as voxels and point clouds often require substantial storage and may lead to the loss of fine-grained geometric and topological details. To address these challenges, we propose SE(3)-BBSCformerGCN, a novel framework for learning shape-aware representations from continuous tubular topological manifolds with equivariance to rotations and translations. Our approach leverages Ball B-Spline Curve (BBSC) to define tubular manifolds and its functional space. We provide a formal mathematical definition and analysis of the resulting manifolds and the BBSC functional space, and incorporate an equivariant mapping that preserves geometric and topological stability. Compared to the point cloud and voxel based representations, our manifold-based formulation significantly reduces data complexity while preserving geometric attributes together with topological features. We validate our method on the branch classification task for Circle of Willis (CoW) on the TopCoW 2024 dataset and the clinical dataset. Our method consistently outperforms voxel and point cloud based baselines in terms of classification performance, generalization ability, convergence speed, and robustness to overfitting.

PDF Details

NeurIPS Conference 2025 Conference Paper

TraffiDent: A Dataset for Understanding the Interplay Between Traffic Dynamics and Incidents

  • Xiaochuan Gou
  • Ziyue Li
  • Tian Lan
  • Junpeng Lin
  • Zhishuai Li
  • Bingyu Zhao
  • Chen Zhang
  • Di Wang

Long-separated research has been conducted on two highly correlated tracks: traffic and incidents. Traffic track witnesses complicating deep learning models, e. g. , to push the prediction a few percent more accurate, and the incident track only studies the incidents alone, e. g. , to infer the incident risk. We, for the first time, spatiotemporally aligned the two tracks in a large-scale region (16, 972 traffic nodes) from year 2022 to 2024: our TraffiDent dataset includes traffic, i. e. , time-series indexes on traffic flow, lane occupancy, and average vehicle speed, and incident, whose records are spatiotemporally aligned with traffic data, with seven different incident classes. Additionally, each node includes detailed physical and policy-level meta-attributes of lanes. Previous datasets typically contain only traffic or incident data in isolation, limiting research to general forecasting tasks. TraffiDent integrates both, enabling detailed analysis of traffic-incident interactions and causal relationships. To demonstrate its broad applicability, we design: (1) post-incident traffic forecasting to quantify the impact of different incidents on traffic indexes; (2) incident classification using traffic indexes to determine the incidents types for precautions measures; (3) global causal analysis among the traffic indexes, meta-attributes, and incidents to give high-level guidance of the interrelations of various factors; (4) local causal analysis within road nodes to examine how different incidents affect the road segments' relations. The dataset is available at https: //xaitraffic. github. io.

PDF Details

AAAI Conference 2024 Conference Paper

Anchoring Path for Inductive Relation Prediction in Knowledge Graphs

  • Zhixiang Su
  • Di Wang
  • Chunyan Miao
  • Lizhen Cui

Aiming to accurately predict missing edges representing relations between entities, which are pervasive in real-world Knowledge Graphs (KGs), relation prediction plays a critical role in enhancing the comprehensiveness and utility of KGs. Recent research focuses on path-based methods due to their inductive and explainable properties. However, these methods face a great challenge when lots of reasoning paths do not form Closed Paths (CPs) in the KG. To address this challenge, we propose Anchoring Path Sentence Transformer (APST) by introducing Anchoring Paths (APs) to alleviate the reliance of CPs. Specifically, we develop a search-based description retrieval method to enrich entity descriptions and an assessment mechanism to evaluate the rationality of APs. APST takes both APs and CPs as the inputs of a unified Sentence Transformer architecture, enabling comprehensive predictions and high-quality explanations. We evaluate APST on three public datasets and achieve state-of-the-art (SOTA) performance in 30 of 36 transductive, inductive, and few-shot experimental settings.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Distilling Autoregressive Models to Obtain High-Performance Non-autoregressive Solvers for Vehicle Routing Problems with Faster Inference Speed

  • Yubin Xiao
  • Di Wang
  • Boyang Li
  • Mingzhao Wang
  • Xuan Wu
  • Changliang Zhou
  • You Zhou

Neural construction models have shown promising performance for Vehicle Routing Problems (VRPs) by adopting either the Autoregressive (AR) or Non-Autoregressive (NAR) learning approach. While AR models produce high-quality solutions, they generally have a high inference latency due to their sequential generation nature. Conversely, NAR models generate solutions in parallel with a low inference latency but generally exhibit inferior performance. In this paper, we propose a generic Guided Non-Autoregressive Knowledge Distillation (GNARKD) method to obtain high-performance NAR models having a low inference latency. GNARKD removes the constraint of sequential generation in AR models while preserving the learned pivotal components in the network architecture to obtain the corresponding NAR models through knowledge distillation. We evaluate GNARKD by applying it to three widely adopted AR models to obtain NAR VRP solvers for both synthesized and real-world instances. The experimental results demonstrate that GNARKD significantly reduces the inference time (4-5 times faster) with acceptable performance drop (2-3%). To the best of our knowledge, this study is first-of-its-kind to obtain NAR VRP solvers from AR ones through knowledge distillation.

PDF Details DOI

AAMAS Conference 2024 Conference Paper

Engaging the Elderly in Exercise with Agents: A Gamified Stationary Bike System for Sarcopenia Management

  • Yang Qiu
  • Ping Chen
  • Huiguo Zhang
  • Bo Huang
  • Di Wang
  • Zhiqi Shen

This paper introduces a portable, gamified exercise system with an embedded agent, specifically designed to aid the elderly in lowerbody workouts using stationary bikes. The system integrates a custom-made Internet of Things (IoT) sensing unit, a gamified application, and an agent-embedded backend platform. By leveraging real-time feedback along with historical user data, the agent actively contributes to exercise safety and adherence by customizing the intensity of workouts and managing break periods. This novel approach aims to make cycling exercise for sarcopenia prevention and intervention more engaging and effective, promoting regular participation and potentially improving health outcomes.

PDF

JMLR Journal 2024 Journal Article

Faster Rates of Differentially Private Stochastic Convex Optimization

  • Jinyan Su
  • Lijie Hu
  • Di Wang

In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) and provide excess population risks for some special classes of functions that are faster than the previous results of general convex and strongly convex functions. In the first part of the paper, we study the case where the population risk function satisfies the Tysbakov Noise Condition (TNC) with some parameter $\theta>1$. Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of $\tilde{O}((\frac{1}{\sqrt{n}}+\frac{d}{n\epsilon})^\frac{\theta}{\theta-1}) $ and $\tilde{O}((\frac{1}{\sqrt{n}}+\frac{\sqrt{d\log(1/\delta)}}{n\epsilon})^\frac{\theta}{\theta-1})$ for $\epsilon$-DP and $(\epsilon, \delta)$-DP, respectively when $\theta\geq 2$, where $n$ is the sample size and $d$ is the dimension of the space. Then we address the inefficiency issue, improve the upper bounds by $\text{Poly}(\log n)$ factors and extend to the case where $\theta\geq \bar{\theta}>1$ for some known $\bar{\theta}$. Next, we show that the excess population risk of population functions satisfying TNC with parameter $\theta\geq 2$ is always lower bounded by $\Omega((\frac{d}{n\epsilon})^\frac{\theta}{\theta-1}) $ and $\Omega((\frac{\sqrt{d\log(1/\delta)}}{n\epsilon})^\frac{\theta}{\theta-1})$ for $\epsilon$-DP and $(\epsilon, \delta)$-DP, respectively, which matches our upper bounds. In the second part, we focus on a special case where the population risk function is strongly convex. Unlike the previous studies, here we assume the loss function is non-negative and the optimal value of population risk is sufficiently small. With these additional assumptions, we propose a new method whose output could achieve an upper bound of $O(\frac{d\log(1/\delta)}{n^2\epsilon^2}+\frac{1}{n^{\tau}})$ and $O(\frac{d^2}{n^2\epsilon^2}+\frac{1}{n^{\tau}})$ for any $\tau> 1$ in $(\epsilon,\delta)$-DP and $\epsilon$-DP model respectively if the sample size $n$ is sufficiently large. These results circumvent their corresponding lower bounds in (Feldman et al., 2020) for general strongly convex functions. Finally, we conduct experiments of our new methods on real-world data. Experimental results also provide new insights into established theories. [abs] [ pdf ][ bib ] &copy JMLR 2024. ( edit, beta )

PDF Details

TCS Journal 2024 Journal Article

Gradient complexity and non-stationary views of differentially private empirical risk minimization

  • Di Wang
  • Jinhui Xu

In this paper, we study the Differentially Private Empirical Risk Minimization (DP-ERM) problem, considering both convex and non-convex loss functions. For cases where DP-ERM involves smooth (strongly) convex loss functions with or without non-smooth regularization, we propose several novel methods. These methods achieve (near) optimal expected excess risks (i. e. , utility bounds) while reducing the gradient complexity compared to existing approaches. When dealing with DP-ERM and smooth convex loss functions in high dimensions, we introduce an algorithm that achieves a superior upper bound with lower gradient complexity than previous solutions. In the second part of the paper, for DP-ERM with non-convex loss functions, we explore both low and high dimensional spaces. In the low dimensional case with a non-smooth regularizer, we extend an existing approach by measuring the utility using the ℓ 2 norm of the projected gradient. Furthermore, we introduce a novel error bound measurement, transitioning from empirical risk to population risk by employing the expected ℓ 2 norm of the gradient. For the high dimensional case, we demonstrate that by measuring utility with the Frank-Wolfe gap, we can bound the utility using the Gaussian Width of the constraint set, instead of the dimensionality (p) of the underlying space. We also show that the advantages of this approach can be achieved by measuring the ℓ 2 norm of the projected gradient. Finally, we reveal that the utility of certain special non-convex loss functions can be reduced to a level (depending only on log ⁡ p ) similar to that of convex loss functions.

Details DOI

AAAI Conference 2024 Conference Paper

Image Content Generation with Causal Reasoning

  • Xiaochuan Li
  • Baoyu Fan
  • Runze Zhang
  • Liang Jin
  • Di Wang
  • Zhenhua Guo
  • Yaqian Zhao
  • Rengang Li

The emergence of ChatGPT has once again sparked research in generative artificial intelligence (GAI). While people have been amazed by the generated results, they have also noticed the reasoning potential reflected in the generated textual content. However, this current ability for causal reasoning is primarily limited to the domain of language generation, such as in models like GPT-3. In visual modality, there is currently no equivalent research. Considering causal reasoning in visual content generation is significant. This is because visual information contains infinite granularity. Particularly, images can provide more intuitive and specific demonstrations for certain reasoning tasks, especially when compared to coarse-grained text. Hence, we propose a new image generation task called visual question answering with image (VQAI) and establish a dataset of the same name based on the classic Tom and Jerry animated series. Additionally, we develop a new paradigm for image generation to tackle the challenges of this task. Finally, we perform extensive experiments and analyses, including visualizations of the generated content and discussions on the potentials and limitations. The code and data are publicly available under the license of CC BY-NC-SA 4.0 for academic and non-commercial usage at: https://github.com/IEIT-AGI/MIX-Shannon/blob/main/projects/VQAI/lgd_vqai.md.

PDF Details DOI

AAAI Conference 2024 Conference Paper

LAFA: Multimodal Knowledge Graph Completion with Link Aware Fusion and Aggregation

  • Bin Shang
  • Yinliang Zhao
  • Jun Liu
  • Di Wang

Recently, an enormous amount of research has emerged on multimodal knowledge graph completion (MKGC), which seeks to extract knowledge from multimodal data and predict the most plausible missing facts to complete a given multimodal knowledge graph (MKG). However, existing MKGC approaches largely ignore that visual information may introduce noise and lead to uncertainty when adding them to the traditional KG embeddings due to the contribution of each associated image to entity is different in diverse link scenarios. Moreover, treating each triple independently when learning entity embeddings leads to local structural and the whole graph information missing. To address these challenges, we propose a novel link aware fusion and aggregation based multimodal knowledge graph completion model named LAFA, which is composed of link aware fusion module and link aware aggregation module. The link aware fusion module alleviates noise of irrelevant visual information by calculating the importance between an entity and its associated images in different link scenarios, and fuses the visual and structural embeddings according to the importance through our proposed modality embedding fusion mechanism. The link aware aggregation module assigns neighbor structural information to a given central entity by calculating the importance between the entity and its neighbors, and aggregating the fused embeddings through linear combination according to the importance. Extensive experiments on standard datasets validate that LAFA can obtain state-of-the-art performance.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation

  • Wentao Jiang
  • Jing Zhang
  • Di Wang
  • Qiming Zhang
  • Zengmao Wang
  • Bo Du

Due to spatial redundancy in remote sensing images, sparse tokens containing rich information are usually involved in self-attention (SA) to reduce the overall token numbers within the calculation, avoiding the high computational cost issue in Vision Transformers. However, such methods usually obtain sparse tokens by hand-crafted or parallel-unfriendly designs, posing a challenge to reach a better balance between efficiency and performance. Different from them, this paper proposes to use learnable meta tokens to formulate sparse tokens, which effectively learn key information meanwhile improving the inference speed. Technically, the meta tokens are first initialized from image tokens via cross-attention. Then, we propose Dual Cross-Attention (DCA) to promote information exchange between image tokens and meta tokens, where they serve as query and key (value) tokens alternatively in a dual-branch structure, significantly reducing the computational complexity compared to self-attention. By employing DCA in the early stages with dense visual tokens, we obtain the hierarchical architecture LeMeViT with various sizes. Experimental results in classification and dense prediction tasks show that LeMeViT has a significant 1. 7 × speedup, fewer parameters, and competitive performance compared to the baseline models, and achieves a better trade-off between efficiency and performance. The code is released at https: //github. com/ViTAE-Transformer/LeMeViT.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

  • Bin Shang
  • Yinliang Zhao
  • Jun Liu
  • Di Wang

Knowledge graph completion (KGC) aims to study the embedding representation to solve the incompleteness of knowledge graphs (KGs). Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been widely used in KGC tasks by capturing neighbor information of entities. However, Both GCNs and GATs based KGC models have their limitations, and the best method is to analyze the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, the representation quality of the embeddings can affect the aggregation of neighbor information (message passing). To address the above limitations, we propose a novel knowledge graph completion model with mixed geometry message and trainable convolutional attention network named MGTCA. Concretely, the mixed geometry message function generates rich neighbor message by integrating spatially information in the hyperbolic space, hypersphere space and Euclidean space jointly. To complete the autonomous switching of graph neural networks (GNNs) and eliminate the necessity of pre-validating the local structure of KGs, a trainable convolutional attention network is proposed by comprising three types of GNNs in one trainable formulation. Furthermore, a mixed geometry scoring function is proposed, which calculates scores of triples by novel prediction function and similarity function based on different geometric spaces. Extensive experiments on three standard datasets confirm the effectiveness of our innovations, and the performance of MGTCA is significantly improved compared to the state-of-the-art approaches.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Perplexity-aware Correction for Robust Alignment with Noisy Preferences

  • Keyi Kong
  • Xilie Xu
  • Di Wang
  • Jingfeng Zhang
  • Mohan Kankanhalli

Alignment techniques are critical in ensuring that large language models (LLMs) output helpful and harmless content by enforcing the LLM-generated content to align with human preferences. However, the existence of noisy preferences (NPs), where the responses are mistakenly labelled as chosen or rejected, could spoil the alignment, thus making the LLMs generate useless and even malicious content. Existing methods mitigate the issue of NPs from the loss perspective by adjusting the alignment loss based on a clean validation dataset. Orthogonal to these loss-oriented methods, we propose perplexity-aware correction (PerpCorrect) from the data perspective for robust alignment which detects and corrects NPs based on the differences between the perplexity of the chosen and rejected responses (dubbed as PPLDiff). Intuitively, a higher PPLDiff indicates a higher probability of the NP because a rejected/chosen response which is mistakenly labelled as chosen/rejected is less preferable to be generated by an aligned LLM, thus having a higher/lower perplexity. PerpCorrect works in three steps: (1) PerpCorrect aligns a surrogate LLM using the clean validation data to make the PPLDiff able to distinguish clean preferences (CPs) and NPs. (2) PerpCorrect further aligns the surrogate LLM by incorporating the reliable clean training data whose PPLDiff is extremely small and reliable noisy training data whose PPLDiff is extremely large after correction to boost the discriminatory power. (3) Detecting and correcting NPs according to the PPLDiff obtained by the aligned surrogate LLM to obtain a denoised training dataset for robust alignment. Comprehensive experiments validate that our proposed PerpCorrect can achieve state-of-the-art alignment performance under NPs. Notably, PerpCorrect demonstrates practical utility by requiring only a modest amount of validation data and being compatible with various alignment techniques. Our code is available at PerpCorrect.

PDF Details DOI

TMLR Journal 2024 Journal Article

Persistent Local Homology in Graph Learning

  • Minghua Wang
  • Yan Hu
  • Ziyun Huang
  • Di Wang
  • Jinhui Xu

In this study, we introduce Persistent Local Homology (PLH) for graphs, a novel method that synergizes persistent homology with local homology to analyze graph structures. We begin by mathematically formalizing PLH, defining it as the application of persistent homology to annular local subgraphs. This foundation paves the way for the development of a computational pipeline, specifically tailored for PLH, which we explore in various graph learning contexts. Despite its utility, a complexity analysis reveals potential computational bottlenecks in PLH application. To address this, we propose Reduced PLH (rPLH), an efficient variant designed to significantly lower computational complexity. Experimental evaluations with rPLH demonstrate its capability to retain the effectiveness of the original PLH while substantially reducing computational demands. The practical utility of PLH and rPLH is further corroborated through comprehensive experiments on both synthetic and real-world datasets, highlighting their broad applicability and potential in diverse analytical scenarios.

PDF Details

AAAI Conference 2024 Conference Paper

Refining Latent Homophilic Structures over Heterophilic Graphs for Robust Graph Convolution Networks

  • Chenyang Qiu
  • Guoshun Nan
  • Tianyu Xiong
  • Wendi Deng
  • Di Wang
  • Zhiyang Teng
  • Lijuan Sun
  • Qimei Cui

Graph convolution networks (GCNs) are extensively utilized in various graph tasks to mine knowledge from spatial data. Our study marks the pioneering attempt to quantitatively investigate the GCN robustness over omnipresent heterophilic graphs for node classification. We uncover that the predominant vulnerability is caused by the structural out-of-distribution (OOD) issue. This finding motivates us to present a novel method that aims to harden GCNs by automatically learning Latent Homophilic Structures over heterophilic graphs. We term such a methodology as LHS. To elaborate, our initial step involves learning a latent structure by employing a novel self-expressive technique based on multi-node interactions. Subsequently, the structure is refined using a pairwisely constrained dual-view contrastive learning approach. We iteratively perform the above procedure, enabling a GCN model to aggregate information in a homophilic way on heterophilic graphs. Armed with such an adaptable structure, we can properly mitigate the structural OOD threats over heterophilic graphs. Experiments on various benchmarks show the effectiveness of the proposed LHS approach for robust GCNs.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Revisiting Differentially Private ReLU Regression

  • Meng Ding
  • Mingxi Lei
  • Liyang Zhu
  • Shaowei Wang
  • Di Wang
  • Jinhui Xu

As one of the most fundamental non-convex learning problems, ReLU regression under differential privacy (DP) constraints, especially in high-dimensional settings, remains a challenging area in privacy-preserving machine learning. Existing results are limited to the assumptions of bounded norm $ \|\mathbf{x}\|_2 \leq 1$, which becomes meaningless with increasing data dimensionality. In this work, we revisit the problem of DP ReLU regression in high-dimensional regimes. We propose two innovative algorithms DP-GLMtron and DP-TAGLMtron that outperform the conventional DPSGD. DP-GLMtron is based on a generalized linear model perceptron approach, integrating adaptive clipping and Gaussian mechanism for enhanced privacy. To overcome the constraints of small privacy budgets in DP-GLMtron, represented by $\widetilde{O}(\sqrt{1/N})$ where $N$ is the sample size, we introduce DP-TAGLMtron, which utilizes a tree aggregation protocol to balance privacy and utility effectively, showing that DP-TAGLMtron achieves comparable performance with only an additional factor of $O(\log N)$ in the utility upper bound. Moreover, our theoretical analysis extends beyond Gaussian-like data distributions to settings with eigenvalue decay, showing how data distribution impacts learning in high dimensions. Notably, our findings suggest that the utility upper bound could be independent of the dimension $d$, even when $d \gg N$. Experiments on synthetic and real-world datasets also validate our results.

PDF Details DOI

AAAI Conference 2024 Conference Paper

SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation

  • Xiaoqi An
  • Lin Zhao
  • Chen Gong
  • Nannan Wang
  • Di Wang
  • Jian Yang

High-resolution representation is essential for achieving good performance in human pose estimation models. To obtain such features, existing works utilize high-resolution input images or fine-grained image tokens. However, this dense high-resolution representation brings a significant computational burden. In this paper, we address the following question: "Only sparse human keypoint locations are detected for human pose estimation, is it really necessary to describe the whole image in a dense, high-resolution manner?" Based on dynamic transformer models, we propose a framework that only uses Sparse High-resolution Representations for human Pose estimation (SHaRPose). In detail, SHaRPose consists of two stages. At the coarse stage, the relations between image regions and keypoints are dynamically mined while a coarse estimation is generated. Then, a quality predictor is applied to decide whether the coarse estimation results should be refined. At the fine stage, SHaRPose builds sparse high-resolution representations only on the regions related to the keypoints and provides refined high-precision human pose estimations. Extensive experiments demonstrate the outstanding performance of the proposed method. Specifically, compared to the state-of-the-art method ViTPose, our model SHaRPose-Base achieves 77.4 AP (+0.5 AP) on the COCO validation set and 76.7 AP (+0.5 AP) on the COCO test-dev set, and infers at a speed of 1.4x faster than ViTPose-Base. Code is available at https://github.com/AnxQ/sharpose.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling

  • Shuaipeng Li
  • Penghao Zhao
  • Hailin Zhang
  • Xingwu Sun
  • Hao Wu
  • Dian Jiao
  • Weiyan Wang
  • Chengjun Liu

In current deep learning tasks, Adam-style optimizers—such as Adam, Adagrad, RMSprop, Adafactor, and Lion—have been widely used as alternatives to SGD-style optimizers. These optimizers typically update model parameters using the sign of gradients, resulting in more stable convergence curves. The learning rate and the batch size are the most critical hyperparameters for optimizers, which require careful tuning to enable effective convergence. Previous research has shown that the optimal learning rate increases linearly (or follows similar rules) with batch size for SGD-style optimizers. However, this conclusion is not applicable to Adam-style optimizers. In this paper, we elucidate the connection between optimal learning rates and batch sizes for Adam-style optimizers through both theoretical analysis and extensive experiments. First, we raise the scaling law between batch sizes and optimal learning rates in the “sign of gradient” case, in which we prove that the optimal learning rate first rises and then falls as the batch size increases. Moreover, the peak value of the surge will gradually move toward the larger batch size as training progresses. Second, we conduct experiments on various CV and NLP tasks and verify the correctness of the scaling law.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Towards Multi-dimensional Explanation Alignment for Medical Classification

  • Lijie Hu
  • Songning Lai
  • Wenshuo Chen
  • Hongru Xiao
  • Hongbin Lin
  • Lu Yu
  • Jingfeng Zhang
  • Di Wang

The lack of interpretability in the field of medical image analysis has significant ethical and legal implications. Existing interpretable methods in this domain encounter several challenges, including dependency on specific models, difficulties in understanding and visualization, and issues related to efficiency. To address these limitations, we propose a novel framework called Med-MICN (Medical Multi-dimensional Interpretable Concept Network). Med-MICN provides interpretability alignment for various angles, including neural symbolic reasoning, concept semantics, and saliency maps, which are superior to current interpretable methods. Its advantages include high prediction accuracy, interpretability across multiple dimensions, and automation through an end-to-end concept labeling process that reduces the need for extensive human training effort when working with new datasets. To demonstrate the effectiveness and interpretability of Med-MICN, we apply it to four benchmark datasets and compare it with baselines. The results clearly demonstrate the superior performance and interpretability of our Med-MICN.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning

  • Zhongzhi Chen
  • Xingwu Sun
  • Xianfeng Jiao
  • Fengzong Lian
  • Zhanhui Kang
  • Di Wang
  • Chengzhong Xu

Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes. Specifically, it creates multiple orthogonal bases for modeling truth by incorporating orthogonal constraints into the probes. Moreover, we introduce Random Peek, a systematic technique considering an extended range of positions within the sequence, reducing the gap between discerning and generating truth features in LLMs. By employing this approach, we improved the truthfulness of Llama-2-7B from 40.8% to 74.5% on TruthfulQA. Likewise, significant improvements are observed in fine-tuned models. We conducted a thorough analysis of truth features using probes. Our visualization results show that orthogonal probes capture complementary truth-related features, forming well-defined clusters that reveal the inherent structure of the dataset.

PDF Details DOI

I&C Journal 2024 Journal Article

Truthful and privacy-preserving generalized linear models

  • Yuan Qiu
  • Jinyan Liu
  • Di Wang

This paper explores estimating Generalized Linear Models (GLMs) when agents are strategic and privacy-conscious. We aim to design mechanisms that encourage truthful reporting, protect privacy, and ensure outputs are close to the true parameters. Initially, we address models with sub-Gaussian covariates and heavy-tailed responses with finite fourth moments, proposing a novel private, closed-form estimator. Our mechanism features: (1) o ( 1 ) -joint differential privacy with high probability; (2) o ( 1 n ) -approximate Bayes Nash equilibrium for ( 1 − o ( 1 ) ) -fraction of agents; (3) o ( 1 ) error in parameter estimation; (4) individual rationality for ( 1 − o ( 1 ) ) of agents; (5) o ( 1 ) payment budget. We then extend our approach to linear regression with heavy-tailed data, using an ℓ 4 -norm shrinkage operator to propose a similar estimator and payment scheme.

Details DOI

NeurIPS Conference 2024 Conference Paper

Truthful High Dimensional Sparse Linear Regression

  • Liyang Zhu
  • Amina Manseur
  • Meng Ding
  • Jinyan Liu
  • Jinhui Xu
  • Di Wang

We study the problem of fitting the high dimensional sparse linear regression model, where the data are provided by strategic or self-interested agents (individuals) who prioritize their privacy of data disclosure. In contrast to the classical setting, our focus is on designing mechanisms that can effectively incentivize most agents to truthfully report their data while preserving the privacy of individual reports. Simultaneously, we seek an estimator which should be close to the underlying parameter. We attempt to solve the problem by deriving a novel private estimator that has a closed-form expression. Based on the estimator, we propose a mechanism which has the following properties via some appropriate design of the computation and payment scheme: (1) the mechanism is $(o(1), O(n^{-\Omega({1})}))$-jointly differentially private, where $n$ is the number of agents; (2) it is an $o(\frac{1}{n})$-approximate Bayes Nash equilibrium for a $(1-o(1))$-fraction of agents to truthfully report their data; (3) the output could achieve an error of $o(1)$ to the underlying parameter; (4) it is individually rational for a $(1-o(1))$ fraction of agents in the mechanism; (5) the payment budget required from the analyst to run the mechanism is $o(1)$. To the best of our knowledge, this is the first study on designing truthful (and privacy-preserving) mechanisms for high dimensional sparse linear regression.

PDF Details DOI

AIIM Journal 2023 Journal Article

Automated atrial fibrillation and ventricular fibrillation recognition using a multi-angle dual-channel fusion network

  • Weiyi Yang
  • Di Wang
  • Wei Fan
  • Gong Zhang
  • Chunying Li
  • Tong Liu

Atrial fibrillation (AFIB) and ventricular fibrillation (VFIB) are two common cardiovascular diseases that cause numerous deaths worldwide. Medical staff usually adopt long-term ECGs as a tool to diagnose AFIB and VFIB. However, since ECG changes are occasionally subtle and similar, visual observation of ECG changes is challenging. To address this issue, we proposed a multi-angle dual-channel fusion network (MDF-Net) to automatically recognize AFIB and VFIB heartbeats in this work. MDF-Net can be seen as the fusion of a task-related component analysis (TRCA)-principal component analysis (PCA) network (TRPC-Net), a canonical correlation analysis (CCA)-PCA network (CPC-Net), and the linear support vector machine-weighted softmax with average (LS-WSA) method. TRPC-Net and CPC-Net are employed to extract deep task-related and correlation features, respectively, from two-lead ECGs, by which multi-angle feature-level information fusion is realized. Since the convolution kernels of the above methods can be directly extracted through TRCA, CCA and PCA technologies, their training time is faster than that of convolutional neural networks. Finally, LS-WSA is employed to fuse the above features at the decision level, by which the classification results are obtained. In distinguishing AFIB and VFIB heartbeats, the proposed method achieved accuracies of 99. 39 % and 97. 17 % in intra- and inter-patient experiments, respectively. In addition, this method performed well on noisy data and extremely imbalanced data, in which abnormal heatbeats are much less than normal heartbeats. Our proposed method has the potential to be used as a diagnostic tool in the clinic.

Details DOI

YNIMG Journal 2023 Journal Article

Deep neural network heatmaps capture Alzheimer’s disease patterns reported in a large meta-analysis of neuroimaging studies

  • Di Wang
  • Nicolas Honnorat
  • Peter T. Fox
  • Kerstin Ritter
  • Simon B. Eickhoff
  • Sudha Seshadri
  • Mohamad Habes

Deep neural networks currently provide the most advanced and accurate machine learning models to distinguish between structural MRI scans of subjects with Alzheimer's disease and healthy controls. Unfortunately, the subtle brain alterations captured by these models are difficult to interpret because of the complexity of these multi-layer and non-linear models. Several heatmap methods have been proposed to address this issue and analyze the imaging patterns extracted from the deep neural networks, but no quantitative comparison between these methods has been carried out so far. In this work, we explore these questions by deriving heatmaps from Convolutional Neural Networks (CNN) trained using T1 MRI scans of the ADNI data set and by comparing these heatmaps with brain maps corresponding to Support Vector Machine (SVM) activation patterns. Three prominent heatmap methods are studied: Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG), and Guided Grad-CAM (GGC). Contrary to prior studies where the quality of heatmaps was visually or qualitatively assessed, we obtained precise quantitative measures by computing overlap with a ground-truth map from a large meta-analysis that combined 77 voxel-based morphometry (VBM) studies independently from ADNI. Our results indicate that all three heatmap methods were able to capture brain regions covering the meta-analysis map and achieved better results than SVM activation patterns. Among them, IG produced the heatmaps with the best overlap with the independent meta-analysis.

Details DOI

JMLR Journal 2023 Journal Article

Generalized Linear Models in Non-interactive Local Differential Privacy with Public Data

  • Di Wang
  • Lijie Hu
  • Huanyu Zhang
  • Marco Gaboardi
  • Jinhui Xu

In this paper, we study the problem of estimating smooth Generalized Linear Models (GLMs) in the Non-interactive Local Differential Privacy (NLDP) model. Unlike its classical setting, our model allows the server to access additional public but unlabeled data. In the first part of the paper, we focus on GLMs. Specifically, we first consider the case where each data record is i.i.d. sampled from a zero-mean multivariate Gaussian distribution. Motivated by the Stein's lemma, we present an $(\epsilon, \delta)$-NLDP algorithm for GLMs. Moreover, the sample complexity of public and private data for the algorithm to achieve an $\ell_2$-norm estimation error of $\alpha$ (with high probability) is ${O}(p \alpha^{-2})$ and $\tilde{O}(p^3\alpha^{-2}\epsilon^{-2})$ respectively, where $p$ is the dimension of the feature vector. This is a significant improvement over the previously known exponential or quasi-polynomial in $\alpha^{-1}$, or exponential in $p$ sample complexities of GLMs with no public data. Then we consider a more general setting where each data record is i.i.d. sampled from some sub-Gaussian distribution with bounded $\ell_1$-norm. Based on a variant of Stein's lemma, we propose an $(\epsilon, \delta)$-NLDP algorithm for GLMs whose sample complexity of public and private data to achieve an $\ell_\infty$-norm estimation error of $\alpha$ is ${O}(p^2\alpha^{-2})$ and $\tilde{O}(p^2\alpha^{-2}\epsilon^{-2})$ respectively, under some mild assumptions and if $\alpha$ is not too small i.e., $\alpha\geq \Omega(\frac{1}{\sqrt{p}})$). In the second part of the paper, we extend our idea to the problem of estimating non-linear regressions and show similar results as in GLMs for both multivariate Gaussian and sub-Gaussian cases. Finally, we demonstrate the effectiveness of our algorithms through experiments on both synthetic and real-world datasets. To our best knowledge, this is the first paper showing the existence of efficient and effective algorithms for GLMs and non-linear regressions in the NLDP model with unlabeled public data. [abs] [ pdf ][ bib ] &copy JMLR 2023. ( edit, beta )

PDF Details

AAAI Conference 2023 Conference Paper

Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformer

  • Zhixiang Su
  • Di Wang
  • Chunyan Miao
  • Lizhen Cui

Recent studies on knowledge graphs (KGs) show that path-based methods empowered by pre-trained language models perform well in the provision of inductive and explainable relation predictions. In this paper, we introduce the concepts of relation path coverage and relation path confidence to filter out unreliable paths prior to model training to elevate the model performance. Moreover, we propose Knowledge Reasoning Sentence Transformer (KRST) to predict inductive relations in KGs. KRST is designed to encode the extracted reliable paths in KGs, allowing us to properly cluster paths and provide multi-aspect explanations. We conduct extensive experiments on three real-world datasets. The experimental results show that compared to SOTA models, KRST achieves the best performance in most transductive and inductive test cases (4 of 6), and in 11 of 12 few-shot test cases.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

On Private and Robust Bandits

  • Yulian Wu
  • Xingyu Zhou
  • Youming Tao
  • Di Wang

We study private and robust multi-armed bandits (MABs), where the agent receives Huber's contaminated heavy-tailed rewards and meanwhile needs to ensure differential privacy. We consider both the finite $k$-th raw moment and the finite $k$-th central moment settings for heavy-tailed rewards distributions with $k\ge 2$. We first present its minimax lower bound, characterizing the information-theoretic limit of regret with respect to privacy budget, contamination level, and heavy-tailedness. Then, we propose a meta-algorithm that builds on a private and robust mean estimation sub-routine \texttt{PRM} that essentially relies on reward truncation and the Laplace mechanism. For the above two different heavy-tailed settings, we give corresponding schemes of \texttt{PRM}, which enable us to achieve nearly-optimal regrets. Moreover, our two proposed truncation-based or histogram-based \texttt{PRM} schemes achieve the optimal trade-off between estimation accuracy, privacy and robustness. Finally, we support our theoretical results and show the effectiveness of our algorithms with experimental studies.

PDF Details

NeurIPS Conference 2023 Conference Paper

SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

  • Di Wang
  • Jing Zhang
  • Bo Du
  • Minqiang Xu
  • Lin Liu
  • Dacheng Tao
  • Liangpei Zhang

The success of the Segment Anything Model (SAM) demonstrates the significance of data-centric machine learning. However, due to the difficulties and high costs associated with annotating Remote Sensing (RS) images, a large amount of valuable RS data remains unlabeled, particularly at the pixel level. In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS totally possesses 105, 090 images and 1, 668, 241 instances, surpassing existing high-resolution RS segmentation datasets in size by several orders of magnitude. It provides object category, location, and instance information that can be used for semantic segmentation, instance segmentation, and object detection, either individually or in combination. We also provide a comprehensive analysis of SAMRS from various aspects. Moreover, preliminary experiments highlight the importance of conducting segmentation pre-training with SAMRS to address task discrepancies and alleviate the limitations posed by limited training data during fine-tuning. The code and dataset will be available at https: //github. com/ViTAE-Transformer/SAMRS

PDF Details

AAAI Conference 2023 Conference Paper

SEAT: Stable and Explainable Attention

  • Lijie Hu
  • Yixin Liu
  • Ninghao Liu
  • Mengdi Huai
  • Lichao Sun
  • Di Wang

Attention mechanism has become a standard fixture in many state-of-the-art natural language processing (NLP) models, not only due to its outstanding performance, but also because it provides plausible innate explanations for neural architectures. However, recent studies show that attention is unstable against randomness and perturbations during training or testing, such as random seeds and slight perturbation of embeddings, which impedes it from being a faithful explanation tool. Thus, a natural question is whether we can find an alternative to vanilla attention, which is more stable and could keep the key characteristics of the explanation. In this paper, we provide a rigorous definition of such an attention method named SEAT (Stable and Explainable ATtention). Specifically, SEAT has the following three properties: (1) Its prediction distribution is close to the prediction of the vanilla attention; (2) Its top-k indices largely overlap with those of the vanilla attention; (3) It is robust w.r.t perturbations, i.e., any slight perturbation on SEAT will not change the attention and prediction distribution too much, which implicitly indicates that it is stable to randomness and perturbations. Furthermore, we propose an optimization method for obtaining SEAT, which could be considered as revising the vanilla attention. Finally, through intensive experiments on various datasets, we compare our SEAT with other baseline methods using RNN, BiLSTM and BERT architectures, with different evaluation metrics on model interpretation, stability and accuracy. Results show that, besides preserving the original explainability and model performance, SEAT is more stable against input perturbations and training randomness, which indicates it is a more faithful explanation.

PDF Details DOI

EAAI Journal 2022 Journal Article

Intelligent weight generation algorithm based on binary isolation tree

  • Di Wang
  • Haoyue Liu
  • Yuming Li

In this paper, in order to make the statistical information of the scoring system more reasonable, a new questionnaire weighting algorithm is proposed to intelligently identify the credibility of survey in this paper. Based on the core idea of the Isolation Forest (iForest), the weights of questionnaire are computationally identified by considering depth of nodes and relative mass between nodes. Using Swee Chuan Tan’s approach of constructing Half-Space Trees, the problem of non-unique weights with repeated calculations is solved and the lower randomness of the result is obtained. In addition, the impact of multiplicity of point on weight is considered by introducing a tiny deviation, and a weight updating method is proposed to deal with the dynamic change of data. The final simulation results demonstrate that the proposed method is defective and accurate, and that the reliability of the questionnaire can be more intelligently recognized in the scoring and evaluation system.

Details DOI

IJCAI Conference 2022 Conference Paper

Private Stochastic Convex Optimization and Sparse Learning with Heavy-tailed Data Revisited

  • Youming Tao
  • Yulian Wu
  • Xiuzhen Cheng
  • Di Wang

In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) with heavy-tailed data, where the gradient of the loss function has bounded moments. Instead of the case where the loss function is Lipschitz or each coordinate of the gradient has bounded second moment studied previously, we consider a relaxed scenario where each coordinate of the gradient only has bounded (1+v)-th moment with some v∈(0, 1]. Firstly, we start from the one dimensional private mean estimation for heavy-tailed distributions. We propose a novel robust and private mean estimator which is optimal. Based on its idea, we then extend to the general d-dimensional space and study DP-SCO with general convex and strongly convex loss functions. We also provide lower bounds for these two classes of loss under our setting and show that our upper bounds are optimal up to a factor of O(Poly(d)). To address the high dimensionality issue, we also study DP-SCO with heavy-tailed gradient under some sparsity constraint (DP sparse learning). We propose a new method and show it is also optimal up to a factor of O(s*), where s* is the underlying sparsity of the constraint.

PDF Details DOI

IJCAI Conference 2022 Conference Paper

Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

  • Di Wang
  • Jinyuan Liu
  • Xin Fan
  • Risheng Liu

Recent learning-based image fusion methods have marked numerous progress in pre-registered multi-modality data, but suffered serious ghosts dealing with misaligned multi-modality data, due to the spatial deformation and the difficulty narrowing cross-modality discrepancy. To overcome the obstacles, in this paper, we present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion (IVIF). Specifically, we propose a Cross-modality Perceptual Style Transfer Network (CPSTN) to generate a pseudo infrared image taking a visible image as input. Benefiting from the favorable geometry preservation ability of the CPSTN, the generated pseudo infrared image embraces a sharp structure, which is more conducive to transforming cross-modality image alignment into mono-modality registration coupled with the structure-sensitive of the infrared image. In this case, we introduce a Multi-level Refinement Registration Network (MRRN) to predict the displacement vector field between distorted and pseudo infrared images and reconstruct registered infrared image under the mono-modality setting. Moreover, to better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM) to adaptively select more meaningful features for fusion in the Dual-path Interaction Fusion Network (DIFN). Extensive experimental results suggest that the proposed method performs superior capability on misaligned cross-modality image fusion.

PDF Details DOI

EAAI Journal 2021 Journal Article

A generalized TODIM-ELECTRE II based integrated decision-making framework for technology selection of energy conservation and emission reduction with unknown weight information

  • Zhengmin Liu
  • Di Wang
  • Xinya Wang
  • Xiaolan Zhao
  • Peide Liu

Energy conservation and emission reduction (ECER) has become an indispensable trend of green life. How to choose a suitable technology for ECER under complex decision-making scenarios has become a key issue faced by traditional industrial enterprises. In this paper, a new integrated decision-making framework is established for technology-selection problems. Firstly, to describe complex linguistic information more accurately and reduce information loss, this framework employs the double hierarchy hesitant fuzzy linguistic term sets (DHHFLTSs) to describe the performance of alternatives. To avoid data redundancy and simplify the computation, we improve the operations of double hierarchy hesitant fuzzy linguistic elements (DHHFLEs) and propose a new comparison method for DHHFLEs. Secondly, based on the proposed normalized projection-based difference measurement, a novel method is proposed to derive the weights of experts regarding each criterion, the group decision-making method based on the best-worst method is also utilized to derive the criteria’s weights via consistent pairwise comparisons. Further, an integrated generalized TODIM (an acronym in Portuguese of interactive and multiple attribute decision making)-ELECTRE II (Elimination and Choice Translating Reality II) method is established to rank alternatives, which has the advantages of dealing with the non-compensatory problem of criteria and considering risk-aversion behavior of experts simultaneously. Subsequently, a case study of emission-reduction technology investment is provided to manifest the practicality and reliability of the given framework. Finally, the preponderance and effectiveness of the proposed method are illuminated through a comparative analysis with several existing representative methods.

Details DOI

AAAI Conference 2021 Conference Paper

CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts

  • Peixiang Zhong
  • Di Wang
  • Pengfei Li
  • Chen Zhang
  • Hao Wang
  • Chunyan Miao

Rationality and emotion are two fundamental elements of humans. Endowing agents with rationality and emotion has been one of the major milestones in AI. However, in the field of conversational AI, most existing models only specialize in one aspect and neglect the other, which often leads to dull or unrelated responses. In this paper, we hypothesize that combining rationality and emotion into conversational agents can improve response quality. To test the hypothesis, we focus on one fundamental aspect of rationality, i. e. , commonsense, and propose CARE, a novel model for commonsense-aware emotional response generation. Specifically, we first propose a framework to learn and construct commonsense-aware emotional latent concepts of the response given an input message and a desired emotion. We then propose three methods to collaboratively incorporate the latent concepts into response generation. Experimental results on two large-scale datasets support our hypothesis and show that our model can produce more accurate and commonsense-aware emotional responses and achieve better human ratings than state-of-the-art models that only specialize in one aspect.

PDF Details

TCS Journal 2021 Journal Article

Differentially private high dimensional sparse covariance matrix estimation

  • Di Wang
  • Jinhui Xu

In this paper, we study the problem of estimating the covariance matrix under differential privacy, where the underlying covariance matrix is assumed to be sparse and of high dimensions. We propose a new method, called DP-Thresholding, to achieve a non-trivial ℓ 2 -norm based error bound whose dependence on the dimension drops to logarithmic instead of polynomial, it is significantly better than the existing ones, which add noise directly to the empirical covariance matrix. We also extend the ℓ 2 -norm based error bound to a general ℓ w -norm based one for any 1 ≤ w ≤ ∞, and show that they share the same upper bound asymptotically. Our approach can be easily extended to local differential privacy. Experiments on the synthetic datasets show results that are consistent with theoretical claims.

Details DOI

IJCAI Conference 2021 Conference Paper

Differentially Private Pairwise Learning Revisited

  • Zhiyu Xue
  • Shaoyang Yang
  • Mengdi Huai
  • Di Wang

Instead of learning with pointwise loss functions, learning with pairwise loss functions (pairwise learning) has received much attention recently as it is more capable of modeling the relative relationship between pairs of samples. However, most of the existing algorithms for pairwise learning fail to take into consideration the privacy issue in their design. To address this issue, previous work studied pairwise learning in the Differential Privacy (DP) model. However, their utilities (population errors) are far from optimal. To address the sub-optimal utility issue, in this paper, we proposed new pure or approximate DP algorithms for pairwise learning. Specifically, under the assumption that the loss functions are Lipschitz, our algorithms could achieve the optimal expected population risk for both strongly convex and general convex cases. We also conduct extensive experiments on real-world datasets to evaluate the proposed algorithms, experimental results support our theoretical analysis and show the priority of our algorithms.

PDF Details DOI

TCS Journal 2021 Journal Article

Inferring ground truth from crowdsourced data under local attribute differential privacy

  • Di Wang
  • Jinhui Xu

Nowadays, crowdsourcing gains an increasing popularity as it can be adopted to solve many challenging question answering tasks that are easy for humans but difficult for computers. Due to the variety in the quality of users, it is important to infer not only the underlying ground truth of these tasks but also the users ability from the answers given by users. This problem is called Ground Truth Inference and has been studied for many years. However, since the answers collected from the users may contain sensitive information, ground truth inference raises serious privacy concern. Due to this reason, the problem of ground truth inference under local differential privacy (LDP) model has been recently studied. However, this problem is still not well understood and even some basic questions have not been solved yet. First, it is still unknown what is the average error of the private estimators to the underlying ground truth. Secondly, we do not know whether we can infer the ability of each user under LDP model and what is the estimation error w. r. t. the underlying users ability. Finally, previous work only shows that their methods have better performance than the private major voting algorithm through experiments. However, there is still no theoretically result which shows this priority formally or mathematically. In this paper, we partially solve these problems by studying the ground truth inference problem under local attribute differential privacy (LADP) model, which is a relaxation of LDP model, and propose a new algorithm called private Dawid-Skene method, which is motivated by the classical Dawid-Skene method. Specifically, we first provide the estimation errors for both ability of users and the ground truth under some assumptions of the problem if the algorithm start with some appropriate initial vector. Moreover, we propose an explicit instance and show that the estimation error of the ground truth achieved by the private major voting algorithm is always greater than the error achieved by our method.

Details DOI

AAAI Conference 2020 Conference Paper

Compact Autoregressive Network

  • Di Wang
  • Feiqing Huang
  • Jingyu Zhao
  • Guodong Li
  • Guangjian Tian

Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the massive amount of parameters in the network leads to expensive computational cost and low learning efficiency. The problem can be alleviated slightly by introducing one more narrow hidden layer to the network, but the sample size required to achieve a certain training error is still substantial. To address this challenge, we rearrange the weight matrices of a linear autoregressive network into a tensor form, and then make use of Tucker decomposition to represent lowrank structures. This leads to a novel compact autoregressive network, called Tucker AutoRegressive (TAR) net. Interestingly, the TAR net can be applied to sequences with longrange dependence since the dimension along the sequential order is reduced. Theoretical studies show that the TAR net improves the learning efficiency, and requires much fewer samples for model training. Experiments on synthetic and real-world datasets demonstrate the promising performance of the proposed compact network.

PDF Details

JMLR Journal 2020 Journal Article

Distributed Kernel Ridge Regression with Communications

  • Shao-Bo Lin
  • Di Wang
  • Ding-Xuan Zhou

This paper focuses on generalization performance analysis for distributed algorithms in the framework of learning theory. Taking distributed kernel ridge regression (DKRR) for example, we succeed in deriving its optimal learning rates in expectation and providing theoretically optimal ranges of the number of local processors. Due to the gap between theory and experiments, we also deduce optimal learning rates for DKRR in probability to essentially reflect the generalization performance and limitations of DKRR. Furthermore, we propose a communication strategy to improve the learning performance of DKRR and demonstrate the power of communications in DKRR via both theoretical assessments and numerical experiments. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2020. ( edit, beta )

PDF Details

JMLR Journal 2020 Journal Article

Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacy

  • Di Wang
  • Marco Gaboardi
  • Adam Smith
  • Jinhui Xu

In this paper, we study the Empirical Risk Minimization (ERM) problem in the non-interactive Local Differential Privacy (LDP) model. Previous research on this problem (Smith et al., 2017) indicates that the sample complexity, to achieve error $\alpha$, needs to be exponentially depending on the dimensionality $p$ for general loss functions. In this paper, we make two attempts to resolve this issue by investigating conditions on the loss functions that allow us to remove such a limit. In our first attempt, we show that if the loss function is $(\infty, T)$-smooth, by using the Bernstein polynomial approximation we can avoid the exponential dependency in the term of $\alpha$. We then propose player-efficient algorithms with $1$-bit communication complexity and $O(1)$ computation cost for each player. The error bound of these algorithms is asymptotically the same as the original one. With some additional assumptions, we also give an algorithm which is more efficient for the server. In our second attempt, we show that for any $1$-Lipschitz generalized linear convex loss function, there is an $(\epsilon, \delta)$-LDP algorithm whose sample complexity for achieving error $\alpha$ is only linear in the dimensionality $p$. Our results use a polynomial of inner product approximation technique. Finally, motivated by the idea of using polynomial approximation and based on different types of polynomial approximations, we propose (efficient) non-interactive locally differentially private algorithms for learning the set of k-way marginal queries and the set of smooth queries. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

AAAI Conference 2020 Conference Paper

Estimating Stochastic Linear Combination of Non-Linear Regressions

  • Di Wang
  • Xiangyu Guo
  • Chaowen Guan
  • Shi Li
  • Jinhui Xu

In this paper we study the problem of estimating stochastic linear combination of non-linear regressions, which has a close connection with many machine learning and statistical models such as non-linear regressions, the Single Index, Multi-index, Varying Coefficient Index Models and Two-layer Neural Networks. Specifically, we first show that with some mild assumptions, if the variate vector x is multivariate Gaussian, then there is an algorithm whose output vectors have 2-norm estimation errors of O( p n ) with high probability, where p is the dimension of x and n is the number of samples. Then we extend our result to the case where x is sub-Gaussian using the zero-bias transformation, which could be seen as a generalization of the classic Stein’s lemma. We also show that with some additional assumptions there is an algorithm whose output vectors have ∞-norm estimation errors of O( 1 √ p + p n ) with high probability. Finally, for both Gaussian and sub-Gaussian cases we propose a faster sub-sampling based algorithm and show that when the sub-sample sizes are large enough then the estimation errors will not be sacrificed by too much. Experiments for both cases support our theoretical results. To the best of our knowledge, this is the first work that studies and provides theoretical guarantees for the stochastic linear combination of non-linear regressions model.

PDF Details

AAAI Conference 2020 Conference Paper

Pairwise Learning with Differential Privacy Guarantees

  • Mengdi Huai
  • Di Wang
  • Chenglin Miao
  • Jinhui Xu
  • Aidong Zhang

Pairwise learning has received much attention recently as it is more capable of modeling the relative relationship between pairs of samples. Many machine learning tasks can be categorized as pairwise learning, such as AUC maximization and metric learning. Existing techniques for pairwise learning all fail to take into consideration a critical issue in their design, i. e. , the protection of sensitive information in the training set. Models learned by such algorithms can implicitly memorize the details of sensitive information, which offers opportunity for malicious parties to infer it from the learned models. To address this challenging issue, in this paper, we propose several differentially private pairwise learning algorithms for both online and offline settings. Specifically, for the online setting, we first introduce a differentially private algorithm (called OnPairStrC) for strongly convex loss functions. Then, we extend this algorithm to general convex loss functions and give another differentially private algorithm (called OnPairC). For the offline setting, we also present two differentially private algorithms (called OffPairStrC and OffPairC) for strongly and general convex loss functions, respectively. These proposed algorithms can not only learn the model effectively from the data but also provide strong privacy protection guarantee for sensitive information in the training set. Extensive experiments on real-world datasets are conducted to evaluate the proposed algorithms and the experimental results support our theoretical analysis.

PDF Details

TCS Journal 2020 Journal Article

Principal Component Analysis in the local differential privacy model

  • Di Wang
  • Jinhui Xu

In this paper, we study the Principal Component Analysis (PCA) problem under the (distributed) non-interactive local differential privacy model. For the low dimensional case (i. e. , p ≪ n ), we show the optimal rate of Θ ( k p n ϵ 2 ) (omitting the eigenvalue terms) for the private minimax risk of the k-dimensional PCA using the squared subspace distance as the measurement, where n is the sample size and ϵ is the privacy parameter. For the high dimensional (i. e. , p ≫ n ) row sparse case, we first give a lower bound of Ω ( k s log ⁡ p n ϵ 2 ) on the private minimax risk, where s is the underlying sparsity parameter. Then we provide an efficient algorithm to achieve the upper bound of O ( s 2 log ⁡ p n ϵ 2 ). Experiments on both synthetic and real world datasets confirm our theoretical guarantees.

Details DOI

TCS Journal 2020 Journal Article

Tight lower bound of sparse covariance matrix estimation in the local differential privacy model

  • Di Wang
  • Jinhui Xu

In this paper, we study the sparse covariance matrix estimation problem in the local differential privacy model, and give a lower bound of Ω ( s 2 log ⁡ p n ϵ 2 ) on the ϵ non-interactive private minimax risk in the metric of squared spectral norm, where s is the row sparsity of the underlying covariance matrix, n is the sample size, and p is the dimensionality of the data. We show that the lower bound is actually tight, as it matches a previous upper bound. Our main technique for achieving this lower bound is a general framework, called General Private Assouad Lemma, which is a considerable generalization of the previous private Assouad lemma and can be used as a general method for bounding the private minimax risk of matrix-related estimation problems.

Details DOI

AAAI Conference 2020 Conference Paper

Towards Interpretation of Pairwise Learning

  • Mengdi Huai
  • Di Wang
  • Chenglin Miao
  • Aidong Zhang

Recently, there are increasingly more attentions paid to an important family of learning problems called pairwise learning, in which the associated loss functions depend on pairs of instances. Despite the tremendous success of pairwise learning in many real-world applications, the lack of transparency behind the learned pairwise models makes it difficult for users to understand how particular decisions are made by these models, which further impedes users from trusting the predicted results. To tackle this problem, in this paper, we study feature importance scoring as a specific approach to the problem of interpreting the predictions of black-box pairwise models. Specifically, we first propose a novel adaptive Shapleyvalue-based interpretation method, based on which a vector of importance scores associated with the underlying features of a testing instance pair can be adaptively calculated with the consideration of feature correlations, and these scores can be used to indicate which features make key contributions to the final prediction. Considering that Shapley-value-based methods are usually computationally challenging, we further propose a novel robust approximation interpretation method for pairwise models. This method is not only much more efficient but also robust to data noise. To the best of our knowledge, we are the first to investigate how to enable interpretation in pairwise learning. Theoretical analysis and extensive experiments demonstrate the effectiveness of the proposed methods.

PDF Details

AAAI Conference 2019 Conference Paper

An Affect-Rich Neural Conversational Model with Biased Attention and Weighted Cross-Entropy Loss

  • Peixiang Zhong
  • Di Wang
  • Chunyan Miao

Natural Language Inference (NLI) is fundamental to many Natural Language Processing (NLP) applications including semantic search and question answering. The NLI problem has gained significant attention due to the release of large scale, challenging datasets. Present approaches to the problem largely focus on learning-based methods that use only textual information in order to classify whether a given premise entails, contradicts, or is neutral with respect to a given hypothesis. Surprisingly, the use of methods based on structured knowledge – a central topic in artificial intelligence – has not received much attention vis-a-vis the NLI problem. While there are many open knowledge bases that contain various types of reasoning information, their use for NLI has not been well explored. To address this, we present a combination of techniques that harness external knowledge to improve performance on the NLI problem in the science questions domain. We present the results of applying our techniques on text, graph, and text-and-graph based models; and discuss the implications of using external knowledge to solve the NLI problem. Our model achieves close to state-of-the-art performance for NLI on the SciTail science questions dataset.

PDF Details

AAAI Conference 2019 Conference Paper

Differentially Private Empirical Risk Minimization with Smooth Non-Convex Loss Functions: A Non-Stationary View

  • Di Wang
  • Jinhui Xu

In this paper, we study the Differentially Private Empirical Risk Minimization (DP-ERM) problem with non-convex loss functions and give several upper bounds for the utility in different settings. We first consider the problem in low-dimensional space. For DP-ERM with non-smooth regularizer, we generalize an existing work by measuring the utility using `2 norm of the projected gradient. Also, we extend the error bound measurement, for the first time, from empirical risk to population risk by using the expected `2 norm of the gradient. We then investigate the problem in high dimensional space, and show that by measuring the utility with Frank-Wolfe gap, it is possible to bound the utility by the Gaussian Width of the constraint set, instead of the dimensionality p of the underlying space. We further demonstrate that the advantages of this result can be achieved by the measure of `2 norm of the projected gradient. A somewhat surprising discovery is that although the two kinds of measurements are quite different, their induced utility upper bounds are asymptotically the same under some assumptions. We also show that the utility of some special non-convex loss functions can be reduced to a level (i. e. , depending only on log p) similar to that of convex loss functions. Finally, we test our proposed algorithms on both synthetic and real world datasets and the experimental results confirm our theoretical analysis.

PDF Details

NeurIPS Conference 2019 Conference Paper

Facility Location Problem in Differential Privacy Model Revisited

  • Yunus Esencayi
  • Marco Gaboardi
  • Shi Li
  • Di Wang

In this paper we study the facility location problem in the model of differential privacy (DP) with uniform facility cost. Specifically, we first show that under the hierarchically well-separated tree (HST) metrics and the super-set output setting that was introduced in Gupta et. al. , there is an $\epsilon$-DP algorithm that achieves an $O(\frac{1}{\epsilon})$(expected multiplicative) approximation ratio; this implies an $O(\frac{\log n}{\epsilon})$ approximation ratio for the general metric case, where $n$ is the size of the input metric. These bounds improve the best-known results given by Gupta et. al. In particular, our approximation ratio for HST-metrics is independent of $n$, and the ratio for general metrics is independent of the aspect ratio of the input metric. On the negative side, we show that the approximation ratio of any $\epsilon$-DP algorithm is lower bounded by $\Omega(\frac{1}{\sqrt{\epsilon}})$, even for instances on HST metrics with uniform facility cost, under the super-set output setting. The lower bound shows that the dependence of the approximation ratio for HST metrics on $\epsilon$ can not be removed or greatly improved. Our novel methods and techniques for both the upper and lower bound may find additional applications.

PDF Details

NeurIPS Conference 2019 Conference Paper

Faster width-dependent algorithm for mixed packing and covering LPs

  • Digvijay Boob
  • Saurabh Sawlani
  • Di Wang

In this paper, we give a faster width-dependent algorithm for mixed packing-covering LPs. Mixed packing-covering LPs are fundamental to combinatorial optimization in computer science and operations research. Our algorithm finds a $1+\eps$ approximate solution in time $O(Nw/ \varepsilon)$, where $N$ is number of nonzero entries in the constraint matrix, and $w$ is the maximum number of nonzeros in any constraint. This algorithm is faster than Nesterov's smoothing algorithm which requires $O(N\sqrt{n}w/ \eps)$ time, where $n$ is the dimension of the problem. Our work utilizes the framework of area convexity introduced in [Sherman-FOCS'17] to obtain the best dependence on $\varepsilon$ while breaking the infamous $\ell_{\infty}$ barrier to eliminate the factor of $\sqrt{n}$. The current best width-independent algorithm for this problem runs in time $O(N/\eps^2)$ [Young-arXiv-14] and hence has worse running time dependence on $\varepsilon$. Many real life instances of mixed packing-covering problems exhibit small width and for such cases, our algorithm can report higher precision results when compared to width-independent algorithms. As a special case of our result, we report a $1+\varepsilon$ approximation algorithm for the densest subgraph problem which runs in time $O(md/ \varepsilon)$, where $m$ is the number of edges in the graph and $d$ is the maximum graph degree.

PDF Details

EAAI Journal 2019 Journal Article

Framework for traffic event detection using Shapelet Transform

  • Ahmed AlDhanhani
  • Ernesto Damiani
  • Rabeb Mizouni
  • Di Wang

Early detection of traffic events is essential for informing Traffic Centers, drivers and intelligent vehicles about incoming dangers or congestion. In this study, a framework based on the shapelets technique for automated incident detection is proposed. Using the shapelets time series classification technique, sub sequences of the time series are generated, which represent patterns of incidents/congestion as well as regular traffic situations. Using such shapelets, our framework is able to detect whether an incident is occurring or not. Application of this approach to real-life data of the London Orbital Motorway (M25) proved that our approach not only has the potential to improve the performance of the classification in terms of false alarm rates or/and accuracy but also provides the human expert with insightful interpretation of the decision made by the event detectors.

Details DOI

IJCAI Conference 2019 Conference Paper

Lower Bound of Locally Differentially Private Sparse Covariance Matrix Estimation

  • Di Wang
  • Jinhui Xu

In this paper, we study the sparse covariance matrix estimation problem in the local differential privacy model, and give a non-trivial lower bound on the non-interactive private minimax risk in the metric of squared spectral norm. We show that the lower bound is actually tight, as it matches a previous upper bound. Our main technique for achieving this lower bound is a general framework, called General Private Assouad Lemma, which is a considerable generalization of the previous private Assouad lemma and can be used as a general method for bounding the private minimax risk of matrix-related estimation problems.

PDF Details

AAAI Conference 2019 Conference Paper

Modelling Autobiographical Memory Loss across Life Span

  • Di Wang
  • Ah-Hwee Tan
  • Chunyan Miao
  • Ahmed A. Moustafa

Neurocomputational modelling of long-term memory is a core topic in computational cognitive neuroscience, which is essential towards self-regulating brain-like AI systems. In this paper, we study how people generally lose their memories and emulate various memory loss phenomena using a neurocomputational autobiographical memory model. Specifically, based on prior neurocognitive and neuropsychology studies, we identify three neural processes, namely overload, decay and inhibition, which lead to memory loss in memory formation, storage and retrieval, respectively. For model validation, we collect a memory dataset comprising more than one thousand life events and emulate the three key memory loss processes with model parameters learnt from memory recall behavioural patterns found in human subjects of different age groups. The emulation results show high correlation with human memory recall performance across their life span, even with another population not being used for learning. To the best of our knowledge, this paper is the first research work on quantitative evaluations of autobiographical memory loss using a neurocomputational model.

PDF Details

IJCAI Conference 2019 Conference Paper

Principal Component Analysis in the Local Differential Privacy Model

  • Di Wang
  • Jinhui Xu

In this paper, we study the Principal Component Analysis (PCA) problem under the (distributed) non-interactive local differential privacy model. For the low dimensional case, we show the optimal rate for the private minimax risk of the k-dimensional PCA using the squared subspace distance as the measurement. For the high dimensional row sparse case, we first give a lower bound on the private minimax risk, . Then we provide an efficient algorithm to achieve a near optimal upper bound. Experiments on both synthetic and real world datasets confirm the theoretical guarantees of our algorithms.

PDF Details

IJCAI Conference 2019 Conference Paper

Privacy-aware Synthesizing for Crowdsourced Data

  • Mengdi Huai
  • Di Wang
  • Chenglin Miao
  • Jinhui Xu
  • Aidong Zhang

Although releasing crowdsourced data brings many benefits to the data analyzers to conduct statistical analysis, it may violate crowd users' data privacy. A potential way to address this problem is to employ traditional differential privacy (DP) mechanisms and perturb the data with some noise before releasing them. However, considering that there usually exist conflicts among the crowdsourced data and these data are usually large in volume, directly using these mechanisms can not guarantee good utility in the setting of releasing crowdsourced data. To address this challenge, in this paper, we propose a novel privacy-aware synthesizing method (i. e. , PrisCrowd) for crowdsourced data, based on which the data collector can release users' data with strong privacy protection for their private information, while at the same time, the data analyzer can achieve good utility from the released data. Both theoretical analysis and extensive experiments on real-world datasets demonstrate the desired performance of the proposed method.

PDF Details

YNICL Journal 2018 Journal Article

Comparison between simultaneously acquired arterial spin labeling and 18F-FDG PET in mesial temporal lobe epilepsy assisted by a PET/MR system and SEEG

  • Yi-He Wang
  • Yang An
  • Xiao-Tong Fan
  • Jie Lu
  • Lian-Kun Ren
  • Peng-Hu Wei
  • Bi-Xiao Cui
  • Jia-Lin Du

Objective: F-FDG), which is invasive, expensive, and radioactive. PET/magnetic resonance (MR) systems have been introduced that allow simultaneous performance of ASL and PET, but comparisons of these techniques with stereoelectroencephalography (SEEG) and comparisons among the treatment outcomes of these techniques are still lacking. Here, we investigate the effectiveness of ASL compared with that of SEEG and their outcomes in localizing mesial temporal lobe epilepsy (MTLE) and assess the correlation between simultaneously acquired PET and ASL. Methods: Between October 2016 and August 2017, we retrospectively studied 12 patients diagnosed with pure unilateral MTLE. We extracted and quantitatively computed values for ASL and PET in the bilateral hippocampus. SEEG findings and outcome were considered the gold standard of lateralization. Finally, the bilateral asymmetry index (AI) was calculated to assess the correlation between PET and ASL. Results: < 0.05). Significance: In our SEEG- and outcome-defined patients with MTLE, ASL could provide significant information during presurgical evaluation, with the hypoperfusion detected with ASL reliably lateralizing MTLE. This non-invasive technique may be used as an alternative diagnostic tool for MTLE lateralization.

Details DOI

NeurIPS Conference 2018 Conference Paper

Empirical Risk Minimization in Non-interactive Local Differential Privacy Revisited

  • Di Wang
  • Marco Gaboardi
  • Jinhui Xu

In this paper, we revisit the Empirical Risk Minimization problem in the non-interactive local model of differential privacy. In the case of constant or low dimensions ($p\ll n$), we first show that if the loss function is $(\infty, T)$-smooth, we can avoid a dependence of the sample complexity, to achieve error $\alpha$, on the exponential of the dimensionality $p$ with base $1/\alpha$ ({\em i. e. ,} $\alpha^{-p}$), which answers a question in \cite{smith2017interaction}. Our approach is based on polynomial approximation. Then, we propose player-efficient algorithms with $1$-bit communication complexity and $O(1)$ computation cost for each player. The error bound is asymptotically the same as the original one. With some additional assumptions, we also give an efficient algorithm for the server. In the case of high dimensions ($n\ll p$), we show that if the loss function is a convex generalized linear function, the error can be bounded by using the Gaussian width of the constrained set, instead of $p$, which improves the one in \cite{smith2017interaction}.

PDF Details

AAAI Conference 2018 Conference Paper

Large Scale Constrained Linear Regression Revisited: Faster Algorithms via Preconditioning

  • Di Wang
  • Jinhui Xu

In this paper, we revisit the large-scale constrained linear regression problem and propose faster methods based on some recent developments in sketching and optimization. Our algorithms combine (accelerated) mini-batch SGD with a new method called two-step preconditioning to achieve an approximate solution with a time complexity lower than that of the state-of-the-art techniques for the low precision case. Our idea can also be extended to the high precision case, which gives an alternative implementation to the Iterative Hessian Sketch (IHS) method with significantly improved time complexity. Experiments on benchmark and synthetic datasets suggest that our methods indeed outperform existing ones considerably in both the low and high precision cases.

PDF Details

IJCAI Conference 2018 Conference Paper

Neural Machine Translation with Key-Value Memory-Augmented Attention

  • Fandong Meng
  • Zhaopeng Tu
  • Yong Cheng
  • Haiyang Wu
  • Junjie Zhai
  • Yuekui Yang
  • Di Wang

Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a novel key-value memory-augmented attention model for NMT, called KVMEMATT. Specifically, we maintain a timely updated keymemory to keep track of attention history and a fixed value-memory to store the representation of source sentence throughout the whole translation process. Via nontrivial transformations and iterative interactions between the two memories, the decoder focuses on more appropriate source word(s) for predicting the next target word at each decoding step, therefore can improve the adequacy of translations. Experimental results on Chinese)English and WMT17 German, English translation tasks demonstrate the superiority of the proposed model.

PDF Details

NeurIPS Conference 2017 Conference Paper

Differentially Private Empirical Risk Minimization Revisited: Faster and More General

  • Di Wang
  • Minwei Ye
  • Jinhui Xu

In this paper we study differentially private Empirical Risk Minimization(ERM) in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms which achieve either optimal or near optimal utility bound with less gradient complexity compared with previous work. For ERM with smooth convex loss function in high-dimension($p\gg n$) setting, we give an algorithm which achieves the upper bound with less gradient complexity than previous ones. At last, we generalize the expected excess empirical risk from convex to Polyak-Lojasiewicz condition and give a tighter upper bound of the utility comparing with the result in \cite{DBLP: journals/corr/ZhangZMW17}.

PDF Details

IJCAI Conference 2017 Conference Paper

Top-k Supervise Feature Selection via ADMM for Integer Programming

  • Mingyu Fan
  • Xiaojun Chang
  • Xiaoqin Zhang
  • Di Wang
  • Liang Du

Recently, structured sparsity inducing based feature selection has become a hot topic in machine learning and pattern recognition. Most of the sparsity inducing feature selection methods are designed to rank all features by certain criterion and then select the k top ranked features, where k is an integer. However, the k top features are usually not the top k features and therefore maybe a suboptimal result. In this paper, we propose a novel supervised feature selection method to directly identify the top k features. The new method is formulated as a classic regularized least squares regression model with two groups of variables. The problem with respect to one group of the variables turn out to be a 0-1 integer programming, which had been considered very hard to solve. To address this, we utilize an efficient optimization method to solve the integer programming, which first replaces the discrete 0-1 constraints with two continuous constraints and then utilizes the alternating direction method of multipliers to optimize the equivalent problem. The obtained result is the top subset with k features under the proposed criterion rather than the subset of k top features. Experiments have been conducted on benchmark data sets to show the effectiveness of proposed method.

PDF Details

AAMAS Conference 2016 Conference Paper

Modeling Autobiographical Memory in Human-Like Autonomous Agents

  • Di Wang
  • Ah-Hwee Tan
  • Chunyan Miao

Although autobiographical memory is an important part of the human mind, there has been little effort on modeling autobiographical memory in autonomous agents. With the motivation of developing human-like intelligence, in this paper, we delineate our approach to enable an agent to maintain memories of its own and to wander in mind. Our model, named Autobiographical Memory-Adaptive Resonance Theory network (AM-ART), is designed to capture autobiographical memories, comprising pictorial snapshots of one’s life experiences together with the associated context, namely time, location, people, activity, and emotion. In terms of both network structure and dynamics, AM-ART coincides with the autobiographical memory model established by the psychologists, which has been supported by neural imaging evidence. Specifically, the bottomup memory search and the top-down memory readout operations of AM-ART replicate how the brain encodes and retrieves autobiographical memories. Furthermore, the wandering in reminiscence function of AM-ART mimics how human wanders in mind. For evaluations, we conducted experiments on a data set collected from the public domain to test the performance of AM-ART in response to exact, partial, and noisy memory retrieval cues. Moreover, our statistical analysis shows that AM-ART can simulate the phenomenon of wandering in reminiscence.

PDF

AAAI Conference 2016 Conference Paper

Semi-Supervised Dictionary Learning via Structural Sparse Preserving

  • Di Wang
  • Xiaoqin Zhang
  • Mingyu Fan
  • Xiuzi Ye

While recent techniques for discriminative dictionary learning have attained promising results on the classification tasks, their performance is highly dependent on the number of labeled samples available for training. However, labeling samples is expensive and time consuming due to the significant human effort involved. In this paper, we present a novel semi-supervised dictionary learning method which utilizes the structural sparse relationships between the labeled and unlabeled samples. Specifically, by connecting the sparse reconstruction coefficients on both the original samples and dictionary, the unlabeled samples can be automatically grouped to the different labeled samples, and the grouped samples share a small number of atoms in the dictionary via mixed 2, pnorm regularization. This makes the learned dictionary more representative and discriminative since the shared atoms are learned by using the labeled and unlabeled samples potentially from the same class. Minimizing the derived objective function is a challenging task because it is non-convex and highly non-smooth. We propose an efficient optimization algorithm to solve the problem based on the block coordinate descent method. Moreover, we have a rigorous proof of the convergence of the algorithm. Extensive experiments are presented to show the superior performance of our method in classification applications.

PDF Details

IJCAI Conference 2015 Conference Paper

An Efficient Classifier Based on Hierarchical Mixing Linear Support Vector Machines

  • Di Wang
  • Xiaoqin Zhang
  • Mingyu Fan
  • Xiuzi Ye

Support vector machines (SVMs) play a very dominant role in data classification due to their good generalization performance. However, they suffer from the high computational complexity in the classification phase when there are a considerable number of support vectors (SVs). Then it is desirable to design efficient algorithms in the classification phase to deal with the datasets of realtime pattern recognition systems. To this end, we propose a novel classifier called HMLSVMs (Hierarchical Mixing Linear Support Vector Machines) in this paper, which has a hierarchical structure with a mixing linear SVMs classifier at each node and predicts the label of a sample using only a few hyperplanes. We also give a generalization error bound for the class of locally linear SVMs (LLSVMs) based on the Rademacher theory, which ensures that overfitting can be effectively avoided. Experimental evaluations shows, while maintaining a comparable classification performance to kernel SVMs (KSVMs), the proposed classifier achieves the high efficiency in the classification stage.

PDF Details

IJCAI Conference 2015 Conference Paper

Multi-Modality Tracker Aggregation: From Generative to Discriminative

  • Xiaoqin Zhang
  • Wei Li
  • Mingyu Fan
  • Di Wang
  • Xiuzi Ye

Visual tracking is an important research topic in computer vision community. Although there are numerous tracking algorithms in the literature, no one performs better than the others under all circumstances, and the best algorithm for a particular dataset may not be known a priori. This motivates a fundamental problem-the necessity of an ensemble learning of different tracking algorithms to overcome their drawbacks and to increase the generalization ability. This paper proposes a multimodality ranking aggregation framework for fusion of multiple tracking algorithms. In our work, each tracker is viewed as a ‘ranker’ which outputs a rank list of the candidate image patches based on its own appearance model in a particular modality. Then the proposed algorithm aggregates the rankings of different rankers to produce a joint ranking. Moreover, the level of expertise for each ‘ranker’ based on the historical ranking results is also effectively used in our model. The proposed model not only provides a general framework for fusing multiple tracking algorithms on multiple modalities, but also provides a natural way to combine the advantages of the generative model based trackers and the the discriminative model based trackers. It does not need to directly compare the output results obtained by different trackers, and such a comparison is usually heuristic. Extensive experiments demonstrate the effectiveness of our work.

PDF Details

YNICL Journal 2015 Journal Article

Performances of diffusion kurtosis imaging and diffusion tensor imaging in detecting white matter abnormality in schizophrenia

  • Jiajia Zhu
  • Chuanjun Zhuo
  • Wen Qin
  • Di Wang
  • Xiaomei Ma
  • Yujing Zhou
  • Chunshui Yu

Diffusion kurtosis imaging (DKI) is an extension of diffusion tensor imaging (DTI), exhibiting improved sensitivity and specificity in detecting developmental and pathological changes in neural tissues. However, little attention was paid to the performances of DKI and DTI in detecting white matter abnormality in schizophrenia. In this study, DKI and DTI were performed in 94 schizophrenia patients and 91 sex- and age-matched healthy controls. White matter integrity was assessed by fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), radial diffusivity (RD), mean kurtosis (MK), axial kurtosis (AK) and radial kurtosis (RK) of DKI and FA, MD, AD and RD of DTI. Group differences in these parameters were compared using tract-based spatial statistics (TBSS) (P AK (20%) > RK (3%) and RD (37%) > FA (24%) > MD (21%) for DKI, and RD (43%) > FA (30%) > MD (21%) for DTI. DKI-derived diffusion parameters (RD, FA and MD) were sensitive to detect abnormality in white matter regions (the corpus callosum and anterior limb of internal capsule) with coherent fiber arrangement; however, the kurtosis parameters (MK and AK) were sensitive to reveal abnormality in white matter regions (the juxtacortical white matter and corona radiata) with complex fiber arrangement. In schizophrenia, the decreased AK suggests axonal damage; however, the increased RD indicates myelin impairment. These findings suggest that diffusion and kurtosis parameters could provide complementary information and they should be jointly used to reveal pathological changes in schizophrenia.

Details DOI

IJCAI Conference 2015 Conference Paper

Semantic Topic Multimodal Hashing for Cross-Media Retrieval

  • Di Wang
  • Xinbo Gao
  • Xiumei Wang
  • Lihuo He

Multimodal hashing is essential to cross-media similarity search for its low storage cost and fast query speed. Most existing multimodal hashing methods embedded heterogeneous data into a common low-dimensional Hamming space, and then rounded the continuous embeddings to obtain the binary codes. Yet they usually neglect the inherent discrete nature of hashing for relaxing the discrete constraints, which will cause degraded retrieval performance especially for long codes. For this purpose, a novel Semantic Topic Multimodal Hashing (STMH) is developed by considering latent semantic information in coding procedure. It first discovers clustering patterns of texts and robust factorizes the matrix of images to obtain multiple semantic topics of texts and concepts of images. Then the learned multimodal semantic features are transformed into a common subspace by their correlations. Finally, each bit of unified hash code can be generated directly by figuring out whether a topic or concept is contained in a text or an image. Therefore, the obtained model by STMH is more suitable for hashing scheme as it directly learns discrete hash codes in the coding process. Experimental results demonstrate that the proposed method outperforms several state-of-the-art methods.

PDF Details

AAAI Conference 2014 Conference Paper

Hybrid Singular Value Thresholding for Tensor Completion

  • Xiaoqin Zhang
  • Zhengyuan Zhou
  • Di Wang
  • Yi Ma

In this paper, we study the low-rank tensor completion problem, where a high-order tensor with missing entries is given and the goal is to complete the tensor. We propose to minimize a new convex objective function, based on log sum of exponentials of nuclear norms, that promotes the low-rankness of unfolding matrices of the completed tensor. We show for the first time that the proximal operator to this objective function is readily computable through a hybrid singular value thresholding scheme. This leads to a new solution to high-order (low-rank) tensor completion via convex relaxation. We show that this convex relaxation and the resulting solution are much more effective than existing tensor completion methods (including those also based on minimizing ranks of unfolding matrices). The hybrid singular value thresholding scheme can be applied to any problem where the goal is to minimize the maximum rank of a set of low-rank matrices.

PDF Details

NeurIPS Conference 2013 Conference Paper

Simultaneous Rectification and Alignment via Robust Recovery of Low-rank Tensors

  • Xiaoqin Zhang
  • Di Wang
  • Zhengyuan Zhou
  • Yi Ma

In this work, we propose a general method for recovering low-rank three-order tensors, in which the data can be deformed by some unknown transformation and corrupted by arbitrary sparse errors. Since the unfolding matrices of a tensor are interdependent, we introduce auxiliary variables and relax the hard equality constraints by the augmented Lagrange multiplier method. To improve the computational efficiency, we introduce a proximal gradient step to the alternating direction minimization method. We have provided proof for the convergence of the linearized version of the problem which is the inner loop of the overall algorithm. Both simulations and experiments show that our methods are more efficient and effective than previous work. The proposed method can be easily applied to simultaneously rectify and align multiple images or videos frames. In this context, the state-of-the-art algorithms RASL'' and "TILT'' can be viewed as two special cases of our work, and yet each only performs part of the function of our method. "

PDF Details
v2026.09.13