Arrow Research search

Author name cluster

Ying Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

69 papers
2 author rows

Possible papers

69

AAAI Conference 2026 Conference Paper

Beyond Immediate Activation: Temporally Decoupled Backdoor Attacks on Time Series Forecasting

  • Zhixin Liu
  • Xuanlin Liu
  • Sihan Xu
  • Yaqiong Qiao
  • Ying Zhang
  • Xiangrui Cai

Existing backdoor attacks on multivariate time series (MTS) forecasting enforce strict temporal and dimensional coupling between triggers and target patterns, requiring synchronous activation at fixed positions across variables. However, realistic scenarios often demand delayed and variable-specific activation. We identify this critical unmet need and propose TDBA, a temporally decoupled backdoor attack framework for MTS forecasting. By injecting triggers that encode the expected location of the target pattern, TDBA enables the activation of the target pattern at any positions within the forecasted data, with the activation position flexibly varying across different variable dimensions. TDBA introduces two core modules: (1) a position-guided trigger generation mechanism that leverages smoothed Gaussian priors to generate triggers that are position-related to the predefined target pattern; and (2) a position-aware optimization module that assigns soft weights based on trigger completeness, pattern coverage, and temporal offset, facilitating targeted and stealthy attack optimization. Extensive experiments on real-world datasets show that TDBA consistently outperforms existing baselines in effectiveness while maintaining good stealthiness. Ablation studies confirm the controllability and robustness of its design.

PDF Details DOI

EAAI Journal 2026 Journal Article

Enhance multi-modal structured representations with open information extraction

  • Jiabao Han
  • Wei Zhong
  • Cheng Wang
  • Ying Zhang
  • Cheng Ding
  • Luwei Luo

Current large-scale vision-language models are expanding their range of applications and have achieved impressive performance in multi-modal tasks. However, the performance of existing models in expressing structured semantic information is of concern, as they have difficulty distinguishing the relationship between subjects and objects in some scene-specific images. This is because the feature learning process in multi-modal task scenarios does not incorporate structured knowledge into the model. In this study, we propose an enhanced end-to-end contrastive language-image pre-training (CLIP) model with open information extraction (OIE-CLIP), which is used to assist in the training of multi-modal models for short texts by integrating structured knowledge representations to enhance the ability of multi-modal representation of structured information. OIE-CLIP leverages the construction of effective negative examples to enhance contrastive learning. In addition, we propose a triple knowledge encoder (TKE) based on the output of open information extraction (OIE) to further boost the structured representation capability of the multi-modal model. To verify the validity of our approach, we pre-trained the model with the above method and downstream experiments in multi-modal tasks. The experimental results show that our method performs best on the public datasets visual genome attribution (VG-Attribution) and visual genome relation (VG-Relation), outperforming the multi-modal state-of-the-art model by 2. 2% and 1. 8%, respectively. Furthermore, our experimental results on the Microsoft Common Objects in Context (MSCOCO) dataset further demonstrate that our method can effectively improve the structured representations.

Details DOI

AAAI Conference 2026 Conference Paper

Grow-on-Demand: Sparse and Adaptive Expert Expansion for Continual Instruction Tuning

  • Ying Zhang
  • Xingyue Guo
  • Yu Zhao
  • Xuhui Sui
  • Baohang Zhou
  • Xinying Qian
  • Xiaojie Yuan

Continual instruction tuning aims to incrementally adapt large language models to new tasks without forgetting previously acquired knowledge. Existing approaches often struggle to balance plasticity and stability. Replay-based methods retrain on historical data, which raises privacy concerns. Architecture-based methods allocate task-specific components, resulting in significant parameter growth. To address this, we consider a structure-sharing strategy that enables parameter reuse across similar tasks and expands only when necessary, avoiding any data replay. Specifically, we introduce Grow-on-Demand (GoD-MoE), a parameter-efficient framework that is based on sparse and adaptive expert module expansion for continual instruction tuning. GoD-MoE inserts multiple LoRA-based experts into attention layers and dynamically activates a small subset of experts for each task. To avoid redundant parameter growth, we develop an Expert Demand Detector that determines whether new experts are added, facilitating adaptive structural sharing and minimizing parameter overhead. We conduct comprehensive experiments on the TRACE benchmark, demonstrating that GoD-MoE achieves state-of-the-art performance. Furthermore, it effectively mitigates catastrophic forgetting and even outperforms several advanced replay-based baselines.

PDF Details DOI

AAAI Conference 2026 System Paper

KnowThyself: An Agentic Assistant for LLM Interpretability

  • Suraj Prasai
  • Mengnan Du
  • Ying Zhang
  • Fan Yang

We develop KnowThyself, an agentic assistant that advances large language model (LLM) interpretability. Existing tools provide useful insights but remain fragmented and code-intensive. KnowThyself consolidates these capabilities into a chat-based interface, where users can upload models, pose natural language questions, and obtain interactive visualizations with guided explanations. At its core, an orchestrator LLM first reformulates user queries, an agent router further directs them to specialized modules, and the outputs are finally contextualized into coherent explanations. This design lowers technical barriers and provides an extensible platform for LLM inspection. By embedding the whole process into a conversational workflow, KnowThyself offers a robust foundation for accessible LLM interpretability.

PDF Details DOI

JBHI Journal 2026 Journal Article

MuGEP: Multiplex Graph-Based Brain Network Modeling for Epileptic Seizure Prediction Using Intracranial EEG

  • Minyu Zhou
  • Yajing Wu
  • Yongqiang Tang
  • Xiaohu Zhou
  • Ying Zhang
  • Runshi Gao
  • Wang Jia
  • Wensheng Zhang

Accurate seizure prediction in advance is crucial for patients with epilepsy, as it helps prevent harm and improve life quality. Intracranial electroencephalogram (iEEG), enabling precise characterization of epileptogenic and propagation networks from deep brain tissue, is a reliable foundation for epilepsy research. While deep learning models have shown success in automating seizure prediction, existing methods often overlook diverse brain network relationships and the rich information within each channel, thereby failing to fully exploit the advantages of iEEG signals. To address these issues, we propose a Multiplex Graph-based brain network modeling framework for Epileptic seizure Prediction (MuGEP) to represent the diverse and fine-grained relationships in the brain network effectively, involving the relationships between amplitude and phase of different frequency bands. Specifically, a specialized multiplex graph called the Multiplex Brain Graph (MBG) is designed for brain network connectivity, which is inspired by Cross-Frequency Coupling (CFC) in neuroscience. In MBG, nodes represent the frequency bands in each channel, and edges are computed following the idea of three types of CFC, resulting in three distinct subgraphs. Further, a novel MBG learning network is proposed to incorporate intra- and inter-subgraph patterns to obtain the final representation leveraging graph convolution networks and a joint fusion module. A sufficient evaluation is conducted on the Kaggle and the SWEC-ETHZ datasets, and the promising results confirm the advantage of MuGEP for iEEG seizure prediction.

Details DOI

AAAI Conference 2026 Conference Paper

STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models

  • Yantao Lu
  • Shiqi Sun
  • Ning Liu
  • Bo Jiang
  • Ying Zhang
  • Jinchao Chen
  • Chenglie Du

Vision-and-Language Navigation (VLN) plays a critical role in tasks of embodied AI, particularly in unseen environments following natural language instructions. Recent advancements leverage large language models (LLMs) to improve the accuracy and generalizability of VLN systems by encoding image sequences as dense token representations. However, this tokenization approach incurs substantial computational overhead due to two key inefficiencies: 1) ego-centric camera views often include navigation-irrelevant re- gions (e.g., sky or distant backgrounds), and 2) high-frame-rate image sequences introduce temporal redundancy. To address these challenges, we propose Spatial-Temporal Efficient Visual Token Pruning (STEP-Nav), a unified frame- work that simultaneously prunes redundant visual tokens and fine-tunes VLN models to preserve navigation performance. In particular, STEP-Nav incorporates a distance- and content-aware token evaluation mechanism to remove irrelevant tokens at the spatial level, along with temporal level similarity-based filtering to reduce redundancy across sequential frames. To ensure pruning does not harm task performance, we introduce a distortion-aware fine-tuning strategy that aligns pruned-token representations with their full-token counterparts while maintaining navigation accuracy. Experiments on the R2R and RxR benchmarks using Navid-CE and NavGPT-2 as base models demonstrate that STEP-Nav preserves over 95% of the performance while reducing 66.7% of tokens, outperforming existing token pruning baselines.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Unlocking Multi-Modal Potentials for Link Prediction on Dynamic Text-Attributed Graphs

  • Yuanyuan Xu
  • Wenjie Zhang
  • Ying Zhang
  • Xuemin Lin
  • Xiwei Xu

Dynamic Text-Attributed Graphs (DyTAGs) are a novel graph paradigm that captures evolving temporal events (edges) alongside rich textual attributes. Existing studies can be broadly categorized into TGNN-driven and LLM-driven approaches, both of which encode textual attributes and temporal structures for DyTAG representation. We observe that DyTAGs inherently comprise three distinct modalities: temporal, textual, and structural, often exhibiting completely disjoint distributions. However, the first two modalities are largely overlooked by existing studies, leading to suboptimal performance. To address this, we propose MoMent, a multi-modal network that explicitly models, integrates, and aligns each modality to learn node representations for link prediction. Given the disjoint nature of the original modality distributions, we first construct modality-specific features and encode them using individual encoders to capture correlations across temporal patterns, semantic context, and local structures. Each encoder generates modality-specific tokens, which are then fused into comprehensive node representations with a theoretical guarantee. To avoid disjoint subspaces of these heterogeneous modalities, we propose a dual-domain alignment loss that first aligns their distributions globally and then fine-tunes coherence at the instance level. This enhances coherent representations from temporal, textual, and structural views. Extensive experiments across seven datasets show that MoMent achieves up to 17.28% accuracy improvement and up to 31x speed-up against eight baselines.

PDF Details DOI

AIIM Journal 2025 Journal Article

A cell-interacting and multi-correcting method for automatic circulating tumor cells detection

  • Xuan Zhang
  • Rensheng Lai
  • Ling Bai
  • Jianxin Ji
  • Ruihao Qin
  • Lihong Jiang
  • Bin Meng
  • Ying Zhang

Sensitive detection of circulating tumor cells (CTCs) from peripheral blood can serve as an effective tool in the early diagnosis and prognosis of cancer. Many methods based on modern object detectors were proposed in recent years for automatic abnormal cells detection in slide images. Although the modes of these methods can also be applied to the CTCs detection, several practical difficulties lead to suboptimal performance of them, such as accurate capture of CTCs in a large number of mixed cells and identification of CTCs and CTC-like cells with similar visual characteristics. Here, we develop a new cell-interacting and multi-correcting detector called CMD, and apply H&E-stained slide images to detect CTCs automatically for the first time. Specifically, the proposed method incorporates two task-oriented novel modules: (1) a self-attention module for aggregating feature interactions between cells and allowing the model to pay more attention to key abnormal cells, (2) a hard sample mining sampler for progressively correcting predictions of cells with ambiguous classification boundaries. Experiments conducted on a multi-center dataset of 1247 annotated slide images confirm the superiority of our method over state-of-the-art cell detection methods. The results of ablation experiment part also prove the effectiveness of two modules. The source codes of this paper are available at https: //github. com/zx333445/CMD.

Details DOI

NeurIPS Conference 2025 Conference Paper

Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks

  • Hongyuan Tao
  • Ying Zhang
  • Zhenhao Tang
  • Hongen Peng
  • Xukun Zhu
  • Bingchang Liu
  • Yingguang Yang
  • Ziyin Zhang

Recent advances in Large Language Models (LLMs) have shown promise in function-level code generation, yet repository-level software engineering tasks remain challenging. Current solutions predominantly rely on proprietary LLM agents, which introduce unpredictability and limit accessibility, raising concerns about data privacy and model customization. This paper investigates whether open-source LLMs can effectively address repository-level tasks without requiring agent-based approaches. We demonstrate this is possible by enabling LLMs to comprehend functions and files within codebases through their semantic information and structural dependencies. To this end, we introduce Code Graph Models (CGMs), which integrate repository code graph structures into the LLM's attention mechanism and map node attributes to the LLM's input space using a specialized adapter. When combined with an agentless graph RAG framework, our approach achieves a 43. 00% resolution rate on the SWE-bench Lite benchmark using the open-source Qwen2. 5-72B model. This performance ranks first among open weight models, second among methods with open-source systems, and eighth overall, surpassing the previous best open-source model-based method by 12. 33%.

PDF Details

AAAI Conference 2025 Conference Paper

CollageNoter: Real-Time and Adaptive Collage Layout Design for Screenshot-Based E-Note-Taking

  • Qiuyun Zhang
  • Bin Guo
  • Lina Yao
  • Xiaotian Qiao
  • Ying Zhang
  • Zhiwen Yu

To enhance the processing of complex multi-modal documents (e.g. e-books, long web pages, etc.), it is an efficient way for users to take digital screenshots of key parts and reorganize them into a new collage E-Note. Existing methods for assisting collage layout design primarily employ a semantic relevance-first strategy, with arranging related contents together. Though capable, it can not ensure the visual readability of screenshots and may conflict with human natural reading patterns. In this paper, we introduce CollageNoter for real-time collage layout design that adapts to various devices (e.g. laptop, tablet, phone, etc.), offering users with visually and cognitively well-organized screenshot-based E-Notes. Specifically, we construct a novel two-stage pipeline for collage design, including 1) readability-first layout generation and 2) cognitive-driven layout adjustment. In addition, to achieve real-time response and adaptive model training, we propose a cascade transformer-based layout generator named CollageFormer and a size-aware collage layout builder for automatic dataset construction. Extensive experimental results have confirmed the effectiveness of our CollageNoter.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs

  • Dongyuan Li
  • Shiyin Tan
  • Ying Zhang
  • Ming Jin
  • Shirui Pan
  • Manabu Okumura
  • Renhe Jiang

Dynamic graph modeling aims to uncover evolutionary patterns in real-world systems, enabling accurate social recommendation and early detection of cancer cells. Inspired by the success of recent state space models in efficiently capturing long-term dependencies, we propose DyG-Mamba by translating dynamic graph modeling into a long-term sequence modeling problem. Specifically, inspired by Ebbinghaus' forgetting curve, we treat the irregular timespans between events as control signals, allowing DyG-Mamba to dynamically adjust the forgetting of historical information. This mechanism ensures effective usage of irregular timespans, thereby improving both model effectiveness and inductive capability. In addition, inspired by Ebbinghaus' review cycle, we redefine core parameters to ensure that DyG-Mamba selectively reviews historical information and filters out noisy inputs, further enhancing the model’s robustness. Through exhaustive experiments on 12 datasets covering dynamic link prediction and node classification tasks, we show that DyG-Mamba achieves state-of-the-art performance on most datasets, while demonstrating significantly improved computational and memory efficiency. Our code is available at https: //github. com/Clearloveyuan/DyG-Mamba.

PDF Details

EAAI Journal 2025 Journal Article

Dynamic modeling and control of pneumatic artificial muscles via Deep Lagrangian Networks and Reinforcement Learning

  • Shuopeng Wang
  • Rixin Wang
  • Yanhui Liu
  • Ying Zhang
  • Lina Hao

Pneumatic artificial muscles (PAMs), as typical soft actuators characterized by hysteresis and nonlinearity, pose a challenging task in modeling and control. This paper proposes a Deep Lagrangian Networks Reinforcement Learning (DeLaNRL) controller that combines deep Lagrangian networks (DeLaN) with reinforcement learning to achieve precise motion control of PAMs. By leveraging the DeLaN model, the dynamic model is constrained to adhere to the Lagrangian first principle, enhancing the model’s compliance with physical constraints. Furthermore, to improve the generality and adaptability of the model to various input data, the Self-scalable tanh (Stan) function is employed as the activation function within the DeLaN model. To validate the effectiveness of the proposed modeling approach, the model is tested on both sampled and unknown motions. The results demonstrate the effectiveness and generalization capability of the DeLaN model with the Stan activation function. Subsequently, the reinforcement learning controller is applied to the learned dynamics model, resulting in control strategies capable of precise motion control. To further demonstrate the effectiveness of the proposed controller, experiments are conducted on both simulation and the experiment platform for reaching and tracking tasks. The simulation results indicate that the control error is less than 0. 91 millimeters, while on the experimental platform, the control error is less than 3. 7 millimeters. These results confirm that the proposed DeLaNRL controller exhibits high control performance.

Details DOI

AAAI Conference 2025 Conference Paper

FedCross: Intertemporal Federated Learning Under Evolutionary Games

  • Jianfeng Lu
  • Ying Zhang
  • Riheng Jia
  • Shuqin Cao
  • Jing Liu
  • Hao Fu

Federated Learning (FL) mitigates privacy leakage in decentralized machine learning by allowing multiple clients to train collaboratively locally. However, dynamic mobile networks with high mobility, intermittent connectivity, and bandwidth limitation severely hinder model updates to the cloud server. Although previous studies have typically addressed user mobility issue through task reassignment or predictive modeling, frequent migrations may result in high communication overhead. Addressing this challenge involves not only dealing with resource constraints, but also finding ways to mitigate the challenges posed by user migrations. We therefore propose a intertemporal incentive framework, FedCross, which ensures the continuity of FL tasks by migrating interrupted training tasks to feasible mobile devices. FedCross comprises two distinct stages: Specifically, in Stage 1, we address the task allocation problem across regions under resource constraints by employing a multi-objective migration algorithm to quantify the optimal task receivers. Moreover, we adopt evolutionary game theory to capture the dynamic decision-making of users, forecasting the evolution of user proportions across different regions to mitigate frequent migrations. In Stage 2, we utilize a procurement auction mechanism to allocate rewards among base stations, ensuring that those providing high-quality models receive optimal compensation. This approach incentivizes sustained user participation, thereby ensuring the overall feasibility of FedCross. Finally, experimental results validate the theoretical soundness of FedCross and demonstrate its significant reduction in communication overhead.

PDF Details DOI

JBHI Journal 2025 Journal Article

Frequency-Aware B-Line and Pleural Line Analysis in Lung Ultrasound Videos

  • Kaihui Yang
  • Guangyu Guo
  • Ying Zhang
  • Linxuan Pang
  • Zhaohui Zheng
  • Ruyu Liu
  • Jin Ding
  • Dingwen Zhang

Accurately identifying B-lines and pleural line (P-line) in lung ultrasound (LUS) videos is valuable for evaluating certain lung conditions. However, manual interpretation remains subjective and highly dependent on operator expertise. Existing deep learning methods often suffer from performance degradation due to speckle noise and motion artifacts. Moreover, the limited availability of LUS video data annotated for multiple diagnostic features such as B-lines and the P-line limits model development. Therefore, this paper introduces ILD-LUS, a new clinical LUS database designed based on interstitial lung disease (ILD) analysis by category labeling, comprising 2, 149 ultrasound videos (193, 410 frames). Also, we construct an external test set based on the public Covid-BLUES dataset for the evaluation of B-lines and P-line recognition in different pulmonary pathologies. Then, we propose a novel video analysis framework that integrates wavelet enhancement with temporal attention modeling. Specifically, we employ a dual-component frequency feature enhancement method using the Discrete Wavelet Transform (DWT), which effectively suppresses noise while preserving important landmarks. Subsequently, an adaptive attention module is introduced to model long-range temporal dependencies and improve dynamic feature representation across consecutive frames. Experimental results show that the proposed method achieves over 94% AUC and 82% ACC for both B-lines and P-line classification on both the ILD-LUS and Covid-BLUES datasets, outperforming existing methods. These findings demonstrate the robustness and generalizability of our approach across different pathological conditions. Overall, the proposed framework shows strong potential for supporting clinical decision-making in LUS analysis. The code is available at https://github.com/KaIi-github/WaveLUS.

Details DOI

JBHI Journal 2025 Journal Article

FROG: A Fine-Grained Spatiotemporal Graph Neural Network With Self-Supervised Guidance for Early Diagnosis of Alzheimer's Disease

  • Shuoyan Zhang
  • Qingmin Wang
  • Min Wei
  • Jiayi Zhong
  • Ying Zhang
  • Ziyan Song
  • Chenyang Li
  • Xiaochen Zhang

Functional magnetic resonance imaging (fMRI) has demonstrated significant potential in the early diagnosis and study of pathological mechanisms of Alzheimer's disease (AD). To fit subtle cross-spatiotemporal interactions and learn pathological features from fMRI, we propose a fine-grained spatiotemporal graph neural network with self-supervised learning (SSL) for diagnosis and biomarker extraction of early AD. First, considering the spatiotemporal interaction of the brain, we design two masks that leverage the spatial correlation and temporal repeatability of fMRI. Afterwards, temporal gated inception convolution and graph scalable inception convolution are proposed for the spatiotemporal autoencoder to enhance subtle cross-spatiotemporal variation and learn noise-suppressed signals. Furthermore, a spatiotemporal scalable cosine error with high selectivity for signal reconstruction is designed in SSL to guide the autoencoder to fit the fine-grained pathological features in an unsupervised manner. A total of 5, 687 samples from four cross-population cohorts are involved. The accuracy of our model was 5. 1% higher than the state-of-the-art models, which included four AD diagnostic models, four SSL strategies, and three multivariate time series models. The neuroimaging biomarkers were precisely localized to the abnormal brain regions, and correlated significantly with the cognitive scale and biomarkers (P $< $ 0. 001). Moreover, the AD progression was reflected through the mask reconstruction error of our SSL strategy. The results demonstrate that our model can effectively capture spatiotemporal and pathological features, and providing a novel and relevant framework for the early diagnosis of AD based on fMRI.

Details DOI

AAAI Conference 2025 Conference Paper

Graph Structure Learning for Spatial-Temporal Imputation: Adapting to Node and Feature Scales

  • Xinyu Yang
  • Yu Sun
  • Xinyang Chen
  • Ying Zhang
  • Xiaojie Yuan

Spatial-temporal data collected across different geographic locations often suffer from missing values, posing challenges to data analysis. Existing methods primarily leverage fixed spatial graphs to impute missing values, which implicitly assume that the spatial relationship is roughly the same for all features across different locations. However, they may overlook the different spatial relationships of diverse features recorded by sensors in different locations. To address this, we introduce the multi-scale Graph Structure Learning framework for spatial-temporal Imputation (GSLI) that dynamically adapts to the heterogeneous spatial correlations. Our framework encompasses node-scale graph structure learning to cater to the distinct global spatial correlations of different features, and feature-scale graph structure learning to unveil common spatial correlation across features within all stations. Integrated with prominence modeling, our framework emphasizes nodes and features with greater significance in the imputation process. Furthermore, GSLI incorporates cross-feature and cross-temporal representation learning to capture spatial-temporal dependencies. Evaluated on six real incomplete spatial-temporal datasets, GSLI showcases the improvement in data imputation and downstream applications.

PDF Details DOI

ICLR Conference 2025 Conference Paper

LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation

  • Mufei Li
  • Viraj Shitole
  • Eli Chien
  • Changhai Man
  • Zhaodong Wang
  • Srinivas
  • Ying Zhang
  • Tushar Krishna

Directed acyclic graphs (DAGs) serve as crucial data representations in domains such as hardware synthesis and compiler/program optimization for computing systems. DAG generative models facilitate the creation of synthetic DAGs, which can be used for benchmarking computing systems while preserving intellectual property. However, generating realistic DAGs is challenging due to their inherent directional and logical dependencies. This paper introduces LayerDAG, an autoregressive diffusion model, to address these challenges. LayerDAG decouples the strong node dependencies into manageable units that can be processed sequentially. By interpreting the partial order of nodes as a sequence of bipartite graphs, LayerDAG leverages autoregressive generation to model directional dependencies and employs diffusion models to capture logical dependencies within each bipartite graph. Comparative analyses demonstrate that LayerDAG outperforms existing DAG generative models in both expressiveness and generalization, particularly for generating large-scale DAGs with up to 400 nodes—a critical scenario for system benchmarking. Extensive experiments on both synthetic and real-world flow graphs from various computing platforms show that LayerDAG generates valid DAGs with superior statistical properties and benchmarking performance. The synthetic DAGs generated by LayerDAG enhance the training of ML-based surrogate models, resulting in improved accuracy in predicting performance metrics of real-world DAGs across diverse computing platforms.

Details

TMLR Journal 2025 Journal Article

On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates

  • Stefano Bruno
  • Ying Zhang
  • Dongyoung Lim
  • Omer Deniz Akyildiz
  • Sotirios Sabanis

We provide full theoretical guarantees for the convergence behaviour of diffusion-based generative models under the assumption of strongly log-concave data distributions while our approximating class of functions used for score estimation is made of Lipschitz continuous functions avoiding any Lipschitzness assumption on the score function. We demonstrate via a motivating example, sampling from a Gaussian distribution with unknown mean, the powerfulness of our approach. In this case, explicit estimates are provided for the associated optimization problem, i.e. score approximation, while these are combined with the corresponding sampling estimates. As a result, we obtain the best known upper bound estimates in terms of key quantities of interest, such as the dimension and rates of convergence, for the Wasserstein-2 distance between the data distribution (Gaussian with unknown mean) and our sampling algorithm. Beyond the motivating example and in order to allow for the use of a diverse range of stochastic optimizers, we present our results using an $L^2$-accurate score estimation assumption, which crucially is formed under an expectation with respect to the stochastic optimizer and our novel auxiliary process that uses only known information. This approach yields the best known convergence rate for our sampling algorithm.

PDF Details

NeurIPS Conference 2025 Conference Paper

RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning

  • Hao Gao
  • Shaoyu Chen
  • Bo Jiang
  • Bencheng Liao
  • Yiang Shi
  • Xiaoyang Guo
  • Yuechuan Pu
  • haoran yin

Existing end-to-end autonomous driving (AD) algorithms typically follow the Imitation Learning (IL) paradigm, which faces challenges such as causal confusion and an open-loop gap. In this work, we propose RAD, a 3DGS-based closed-loop Reinforcement Learning (RL) framework for end-to-end Autonomous Driving. By leveraging 3DGS techniques, we construct a photorealistic digital replica of the real physical world, enabling the AD policy to extensively explore the state space and learn to handle out-of-distribution scenarios through large-scale trial and error. To enhance safety, we design specialized rewards to guide the policy in effectively responding to safety-critical events and understanding real-world causal relationships. To better align with human driving behavior, we incorporate IL into RL training as a regularization term. We introduce a closed-loop evaluation benchmark consisting of diverse, previously unseen 3DGS environments. Compared to IL-based methods, RAD achieves stronger performance in most closed-loop metrics, particularly exhibiting a 3× lower collision rate. Abundant closed-loop results are presented in the supplementary material. Code is available at https: //github. com/hustvl/RAD for facilitating future research.

PDF Details

JBHI Journal 2025 Journal Article

Swallow-PPG: Photoplethysmography Templates for Comprehensive Temporal Analysis of Swallowing Anatomical Actions

  • Ying Zhang
  • Junjie Li
  • Ping Wang
  • Huaiyu Zhu
  • Bo Wang
  • Wei Luo
  • Yun Pan

In clinical practice, Videofluoroscopic Swallowing Study (VFSS) is commonly used to monitor the activity of anatomical structures during swallowing. However, it is limited by ionizing radiation exposure, adverse effects of barium contrast agents, and the high cost of specialized equipment. In this study, we propose a framework for analyzing swallowing behaviors in photoplethysmography (PPG) waveforms, which includes generalizing the manifestation of swallowing in PPG (i. e. , swallowing templates generation) and conducting comprehensive temporal analysis of swallowing anatomical actions (TASAA). For swallowing templates generation, we cluster and average the samples to obtain waveforms of templates, followed by conducting shape-based mapping and averaging on 28 time indicators to derive template unified time indicators (TUTIs). For comprehensive TASAA, we leverage templates waveforms and TUTIs to estimate time indicators based on the mapping relationship between samples and their respective templates. We evaluate the proposed framework on 357 swallowing PPG samples from 41 elderly subjects. The average relative error across all time indicators is 0. 123, and 6 indicators notably excel with errors below 0. 1. The proposed template-based swallowing analysis framework is expected to become a low-cost and non-ionizing alternative to VFSS for comprehensive TASAA.

Details DOI

ICML Conference 2025 Conference Paper

TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Vision

  • Shaobin Zhuang
  • Yiwei Guo
  • Yanbo Ding
  • Kunchang Li 0002
  • Xinyuan Chen
  • Yaohui Wang 0001
  • Fangyikang Wang
  • Ying Zhang

Diffusion models have driven the advancement of vision generation over the past years. However, it is often difficult to apply these large models in downstream tasks, due to massive fine-tuning cost. Recently, Low-Rank Adaptation (LoRA) has been applied for efficient tuning of diffusion models. Unfortunately, the capabilities of LoRA-tuned diffusion models are limited, since the same LoRA is used for different timesteps of the diffusion process. To tackle this problem, we introduce a general and concise TimeStep Master (TSM) paradigm with two key fine-tuning stages. In the fostering stage (1-stage), we apply different LoRAs to fine-tune the diffusion model at different timestep intervals. This results in different TimeStep LoRA experts that can effectively capture different noise levels. In the assembling stage (2-stage), we design a novel asymmetrical mixture of TimeStep LoRA experts, via core-context collaboration of experts at multi-scale intervals. For each timestep, we leverage TimeStep LoRA expert within the smallest interval as the core expert without gating, and use experts within the bigger intervals as the context experts with time-dependent gating. Consequently, our TSM can effectively model the noise level via the expert in the finest interval, and adaptively integrate contexts from the experts of other scales, boosting the versatility of diffusion models. To show the effectiveness of our TSM paradigm, we conduct extensive experiments on three typical and popular LoRA-related tasks of diffusion models, including domain adaptation, post-pretraining, and model distillation. Our TSM achieves the state-of-the-art results on all these tasks, throughout various model structures (UNet, DiT and MM-DiT) and visual data modalities (Image, Video), showing its remarkable generalization capacity.

Details

IJCAI Conference 2025 Conference Paper

Tree-of-AdEditor: Heuristic Tree Reasoning for Automated Video Advertisement Editing with Large Language Model

  • Yuqi Zhang
  • Bin Guo
  • Nuo Li
  • Ying Zhang
  • Shijie Wang
  • Zhiwen Yu
  • Qing Li

Video advertising has become a popular marketing strategy on e-commerce platforms, requiring high-level semantic reasoning like selling point discovery, narrative organization. Previous rule-based methods struggle with these complex tasks, and learning-based approaches demand large datasets and high training costs. Recently, Large Language Models have opened incredible opportunities for advancing intelligent video advertisement editing. However, Input-output (IO) prompting and Chain-of-Thought (CoT) struggle to adapt to the nonlinear thinking hierarchy of video editing, where editors iteratively select shots or revert them to explore potential editing solutions. While Tree-of-Thought (ToT) offers a conceptual structure that mirrors this hierarchy, it falls short in aligning with effective video advertising strategies and lacks robust fact-checking mechanisms. To address these, we propose a novel framework, Tree-of-AdEditor (ToAE), which constructs a reasoning tree to mimic human editors, and incorporates domain-specific theories and heuristic fact-checking to identify optimal editing solutions. Specifically, motivated by effective advertisement principles, we develop a "local-global" mechanism to guide LLM in both the shot level and sequence level decision-making. We introduce a visual incoherence pruning module to provide external heuristic fact-checking, ensuring visual attractiveness and reducing computation costs. Quantitative experiments and expert evaluation demonstrate the superiority of our method compared to baselines.

PDF Details DOI

EAAI Journal 2025 Journal Article

WB-YOLO: An efficient wild bat detection method for ecological monitoring in complex environments

  • Yang Wang
  • Chang Ma
  • Chuanxin Zhao
  • Huijuan Xia
  • Congxi Chen
  • Ying Zhang

The study of bat species and their distribution is vital for understanding the origins and transmission pathways of epidemic diseases. However, the detection of wild bats faces significant challenges due to their complex natural habitats and frequent occlusions caused by social behavior. To address these issues, we propose an object detection method based on the improved You Only Look Once version 7 (YOLOv7) to achieve efficient wild bat detection (WB-YOLO). This method integrates a Vision Transformer encoder module to improve global context integration, adopts deformable convolution, and optimizes the spatial pyramid pooling structure for effective multi-scale feature fusion while reducing computational complexity. Furthermore, a hybrid attention mechanism is introduced to capture both spatial and channel information, enhancing robustness in complex environments. Experimental results in a data set of wild bat images collected in Anhui Province demonstrate that WB-YOLO achieves a precision of 90. 7%, a recall of 89. 0%, and a mean average precision (mAP) of 94. 7%, significantly outperforming other deep learning models in detecting bats in complex scenes and under occlusion. Our approach offers an efficient and accurate solution for the detection of wild bats in real time, with potential applications in ecological research and disease prevention. Code and data related to this work are publicly available at https: //github. com/macandzzz/WB-YOLO.

Details DOI

IROS Conference 2024 Conference Paper

6-DoF Grasp Detection in Clutter with Enhanced Receptive Field and Graspable Balance Sampling

  • Hanwen Wang
  • Ying Zhang
  • Yunlong Wang
  • Jian Li

6-DoF grasp detection of small-scale grasps is crucial for robots to perform specific tasks. This paper focuses on enhancing the recognition capability of small-scale grasping, aiming to improve the overall accuracy of grasping prediction results and the generalization ability of the network. We propose an enhanced receptive field method that includes a multi-radii cylinder grouping module and a passive attention module. This method enhances the receptive field area within the graspable space and strengthens the learning of graspable features. Additionally, we design a graspable balance sampling module based on a 3D segmentation network, which enables the network to focus on features of small objects, thereby improving the recognition capability of small-scale grasping. Our network achieves state-of-the-art performance on the GraspNet-1Billion dataset, with an overall improvement of approximately 10% in average precision@k (AP). Furthermore, we deployed our grasp detection model on pybullet grasping platform and in real-world scenarios, which validates the effectiveness of our method.

Details

EAAI Journal 2024 Journal Article

A novel health indicator by dominant invariant subspace on Grassmann manifold for state of health assessment of lithium-ion battery

  • Ying Zhang
  • Yan-Fu Li
  • Ming Zhang
  • Huan Wang

The precise estimation of the state of health (SoH) in Lithium-ion batteries (LiBs) relies heavily on a reliable health indicator (HI). Conventional indicators are often constructed by directly concatenating features from multiple sources. It overlooks significant non-linear and correlative information inherent in raw signals. To address this limitation, this paper introduces an innovative approach for SoH estimation in LiBs. Deep features extracted from signals of various sensors are obtained using denoising auto-encoders (DAEs). Then the dominant invariant subspaces (DIS) are calculated through the non-linear transformation of multi-source features on the Grassmann manifold. It can preserve essential and robust characteristics. The health indicator quantifies the geodesic distance of DIS using a projection metric. It provides a more comprehensive inclusion of nonlinear and correlation information. Consequently, this indicator offers heightened precision in discerning differences in health states. Validation of the proposed method is conducted using the NASA dataset. The result demonstrates its effectiveness on the SoH assessment and superiority to the state-of-the-art method.

Details DOI

NeurIPS Conference 2024 Conference Paper

AlterMOMA: Fusion Redundancy Pruning for Camera-LiDAR Fusion Models with Alternative Modality Masking

  • Shiqi Sun
  • Yantao Lu
  • Ning Liu
  • Bo Jiang
  • Jinchao Chen
  • Ying Zhang

Camera-LiDAR fusion models significantly enhance perception performance in autonomous driving. The fusion mechanism leverages the strengths of each modality while minimizing their weaknesses. Moreover, in practice, camera-LiDAR fusion models utilize pre-trained backbones for efficient training. However, we argue that directly loading single-modal pre-trained camera and LiDAR backbones into camera-LiDAR fusion models introduces similar feature redundancy across modalities due to the nature of the fusion mechanism. Unfortunately, existing pruning methods are developed explicitly for single-modal models, and thus, they struggle to effectively identify these specific redundant parameters in camera-LiDAR fusion models. In this paper, to address the issue above on camera-LiDAR fusion models, we propose a novelty pruning framework Alternative Modality Masking Pruning (AlterMOMA), which employs alternative masking on each modality and identifies the redundant parameters. Specifically, when one modality parameters are masked (deactivated), the absence of features from the masked backbone compels the model to reactivate previous redundant features of the other modality backbone. Therefore, these redundant features and relevant redundant parameters can be identified via the reactivation process. The redundant parameters can be pruned by our proposed importance score evaluation function, Alternative Evaluation (AlterEva), which is based on the observation of the loss changes when certain modality parameters are activated and deactivated. Extensive experiments on the nuScene and KITTI datasets encompassing diverse tasks, baseline models, and pruning algorithms showcase that AlterMOMA outperforms existing pruning methods, attaining state-of-the-art performance.

PDF Details DOI

EAAI Journal 2024 Journal Article

An integrated hesitant 2-tuple linguistic Pythagorean fuzzy decision-making method for single-pilot operations mechanism evaluation

  • Fei Gao
  • Ying Zhang
  • Yijia Li
  • Wenhao Bi

Single-pilot operations (SPO), i. e. , one pilot on board the flight in charge of all the operations, has received extensive attention in recent years. SPO involves several participants including the pilot, the autopilot, and the ground operator, and it is of great significance to determine the roles of these participants under the SPO mechanism. However, the evaluation of SPO mechanism has received little attention. Aiming at providing a reliable approach for SPO mechanism evaluation under uncertainty, this paper proposes a novel hesitant 2-tuple linguistic Pythagorean fuzzy decision-making method based on hesitant fuzzy linguistic term sets (HFLTS), 2-tuple linguistic information, Pythagorean fuzzy sets (PFS), and VIKOR method to handle the SPO mechanism evaluation problem. Firstly, based on the analysis of SPO, the evaluation criteria system for SPO mechanism is established. Then, considering the uncertain information in the SPO mechanism evaluation process, the HFLTS is used to represent the linguistic judgments of experts under uncertainty. Next, the HFLTSs of different experts are converted into 2-tuple linguistic information and PFSs to model the overall judgments of different criteria. Finally, the extended VIKOR method is adopted to evaluate and rank different SPO mechanisms. A case study of SPO mechanism evaluation for the ground proximity warning system (GPWS) is used to demonstrate the effectiveness and feasibility of the proposed method, and the SPO mechanism that the autopilot being responsible for GPWS is evaluated by the proposed method to be optimal. Sensitivity and comparison analyses further confirm that the proposed method could provide reliable and reasonable SPO mechanism evaluation results. In conclusion, the proposed method presents a novel and effective way to evaluate and select appropriate SPO mechanisms.

Details DOI

NeurIPS Conference 2024 Conference Paper

Can We Leave Deepfake Data Behind in Training Deepfake Detector?

  • Jikang Cheng
  • Zhiyuan Yan
  • Ying Zhang
  • Yuhao Luo
  • Zhongyuan Wang
  • Chen Li

The generalization ability of deepfake detectors is vital for their applications in real-world scenarios. One effective solution to enhance this ability is to train the models with manually-blended data, which we termed ''blendfake'', encouraging models to learn generic forgery artifacts like blending boundary. Interestingly, current SoTA methods utilize blendfake $\textit{without}$ incorporating any deepfake data in their training process. This is likely because previous empirical observations suggest that vanilla hybrid training (VHT), which combines deepfake and blendfake data, results in inferior performance to methods using only blendfake data (so-called "1+1<2"). Therefore, a critical question arises: Can we leave deepfake behind and rely solely on blendfake data to train an effective deepfake detector? Intuitively, as deepfakes also contain additional informative forgery clues ($\textit{e. g. ,}$ deep generative artifacts), excluding all deepfake data in training deepfake detectors seems counter-intuitive. In this paper, we rethink the role of blendfake in detecting deepfakes and formulate the process from "real to blendfake to deepfake" to be a $\textit{progressive transition}$. Specifically, blendfake and deepfake can be explicitly delineated as the oriented pivot anchors between "real-to-fake" transitions. The accumulation of forgery information should be oriented and progressively increasing during this transition process. To this end, we propose an $\underline{O}$riented $\underline{P}$rogressive $\underline{R}$egularizor (OPR) to establish the constraints that compel the distribution of anchors to be discretely arranged. Furthermore, we introduce feature bridging to facilitate the smooth transition between adjacent anchors. Extensive experiments confirm that our design allows leveraging forgery information from both blendfake and deepfake effectively and comprehensively. Code is available at https: //github. com/beautyremain/ProDet.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

  • Jay Shah
  • Ganesh Bikshandi
  • Ying Zhang
  • Vijay Thakkar
  • Pradeep Ramani
  • Tri Dao

Attention, as a core layer of the ubiquitous Transformer architecture, is the bottleneck for large language models and long-context applications. elaborated an approach to speed up attention on GPUs through minimizing memory reads/writes. However, it has yet to take advantage of new capabilities present in recent hardware, with FlashAttention-2 achieving only 35% utilization on the H100 GPU. We develop three main techniques to speed up attention on Hopper GPUs: exploiting asynchrony of the Tensor Cores and TMA to (1) overlap overall computation and data movement via warp-specialization and (2) interleave block-wise matmul and softmax operations, and (3) block quantization and incoherent processing that leverages hardware support for FP8 low-precision. We demonstrate that our method, FlashAttention-3, achieves speedup on H100 GPUs by 1. 5-2. 0$\times$ with BF16 reaching up to 840 TFLOPs/s (85\% utilization), and with FP8 reaching 1. 3 PFLOPs/s. We validate that FP8 FlashAttention-3 achieves 2. 6$\times$ lower numerical error than a baseline FP8 attention.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Hypergraph Self-supervised Learning with Sampling-efficient Signals

  • Fan Li
  • Xiaoyang Wang
  • Dawei Cheng
  • Wenjie Zhang
  • Ying Zhang
  • Xuemin Lin

Self-supervised learning (SSL) provides a promising alternative for representation learning on hypergraphs without costly labels. However, existing hypergraph SSL models are mostly based on contrastive methods with the instance-level discrimination strategy, suffering from two significant limitations: (1) They select negative samples arbitrarily, which is unreliable in deciding similar and dissimilar pairs, causing training bias. (2) They often require a large number of negative samples, resulting in expensive computational costs. To address the above issues, we propose SE-HSSL, a hypergraph SSL framework with three sampling-efficient self-supervised signals. Specifically, we introduce two sampling-free objectives leveraging the canonical correlation analysis as the node-level and group-level self-supervised signals. Additionally, we develop a novel hierarchical membership-level contrast objective motivated by the cascading overlap relationship in hypergraphs, which can further reduce membership sampling bias and improve the efficiency of sample utilization. Through comprehensive experiments on 7 real-world hypergraphs, we demonstrate the superiority of our approach over the state-of-the-art method in terms of both effectiveness and efficiency.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Learning Time Slot Preferences via Mobility Tree for Next POI Recommendation

  • Tianhao Huang
  • Xuan Pan
  • Xiangrui Cai
  • Ying Zhang
  • Xiaojie Yuan

Next Point-of-Interests (POIs) recommendation task aims to provide a dynamic ranking of POIs based on users' current check-in trajectories. The recommendation performance of this task is contingent upon a comprehensive understanding of users' personalized behavioral patterns through Location-based Social Networks (LBSNs) data. While prior studies have adeptly captured sequential patterns and transitional relationships within users' check-in trajectories, a noticeable gap persists in devising a mechanism for discerning specialized behavioral patterns during distinct time slots, such as noon, afternoon, or evening. In this paper, we introduce an innovative data structure termed the ``Mobility Tree'', tailored for hierarchically describing users' check-in records. The Mobility Tree encompasses multi-granularity time slot nodes to learn user preferences across varying temporal periods. Meanwhile, we propose the Mobility Tree Network (MTNet), a multitask framework for personalized preference learning based on Mobility Trees. We develop a four-step node interaction operation to propagate feature information from the leaf nodes to the root node. Additionally, we adopt a multitask training strategy to push the model towards learning a robust representation. The comprehensive experimental results demonstrate the superiority of MTNet over eleven state-of-the-art next POI recommendation models across three real-world LBSN datasets, substantiating the efficacy of time slot preference learning facilitated by Mobility Tree.

PDF Details DOI

EAAI Journal 2024 Journal Article

UDANet: An unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning

  • Ying Zhang
  • Hong-Mei Sun
  • Rui Yang
  • Rui-Sheng Jia

Unsupervised domain adaptive vehicle density estimation aims to transfer the knowledge learned from the labeled source domain to the unlabeled target domain, which has received extensive attention due to its practicality and effectiveness. However, the existing methods often rely on a large amount of expensive labeled data, and ignore the large differences between different data domains, making the network trained in one scene difficult to apply to the target scene. To solve these problems, this paper proposes an unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning (UDANet), which effectively extends the model to unlabeled target domain data by adding two domain adaptive components to the pre-trained backbone. Specifically, a multi-level feature discriminator is designed to align vehicle features from the image level and refine features from the pixel level by joint adversarial learning, thus achieving cross-domain transfer of features from the source domain to the target domain. In addition, to address the shared convolutional structure that incorrectly highlights irrelevant information, a feature correction module is designed to improve the adaptability of the network to target domain data by re-modeling the source and target data according to channel independence. Several well-known backbones are used to prove the superiority of the backbone and the mobility of the two domain adaptive components. The domain adaptation experimental results on four vehicle datasets show that UDANet achieves high estimation accuracy compared to current state-of-the-art unsupervised methods, and is suitable for the task of vehicle density estimation in complex scenes.

Details DOI

JBHI Journal 2023 Journal Article

TransFusionNet: Semantic and Spatial Features Fusion Framework for Liver Tumor and Vessel Segmentation Under JetsonTX2

  • Xun Wang
  • Xudong Zhang
  • Gan Wang
  • Ying Zhang
  • Xin Shi
  • Huanhuan Dai
  • Min Liu
  • Zixuan Wang

Liver cancer is one of the most common malignant diseases worldwide. Segmentation and reconstruction of liver tumors and vessels in CT images can provide convenience for physicians in preoperative planning and surgical intervention. In this paper, we introduced a TransFusionNet framework, which consists of a semantic feature extraction module, a local spatial feature extraction module, an edge feature extraction module, and a multi-scale feature fusion module to achieve fine-grained segmentation of liver tumors and vessels. In addition, we applied the transfer learning approach to pre-train using public datasets and then fine-tune the model to further improve the fitting effect. Furthermore, we proposed an intelligent quantization scheme to compress the model weights and achieved high performance inference on JetsonTX2. The TransFusionNet framework achieved mean IoU of 0. 854 in vessel segmentation task, and achieved mean IoU of 0. 927 in liver tumor segmentation task. When profiling the Computational Performance of the quantized inference, our quantized model achieved 4TFLOPs on Node with NVIDIA RTX3090 and 132GFLOPs on JetsonTX2. This unprecedented segmentation effect solves the accuracy and performance bottleneck of automated segmentation to a certain extent.

Details DOI

EAAI Journal 2022 Journal Article

Aerial combat maneuvering policy learning based on confrontation demonstrations and dynamic quality replay

  • Dongyuan Hu
  • Rennong Yang
  • Ying Zhang
  • Longfei Yue
  • Mengda Yan
  • Jialiang Zuo
  • Xiaoru Zhao

Unmanned Combat Aerial Vehicles (UCAVs) is becoming a crucial platform to perform dangerous missions with the ability of autonomy and intelligence in making decisions. Recently, many researches have focused on UCAV air-to-air combat mission and attempted to solve maneuvering policy using deep reinforcement learning (DRL). Yet, previous studies on the sequential decision problem often have limitations due to the complexity of combat environment, overdependence on expert knowledge, and low learning efficiency from large-scale exploration space. In view of the lacks, the paper aims to automatically formulate counter maneuvers with the application of DRL and proposes a novel dynamic quality replay (DQR) method which is capable of guiding agent learn tactical policy from historical data efficiently, and can get rid of the dependence on traditional expert knowledge. Firstly, in order to accelerate model convergence, confrontation demonstrations generated with traditional algorithms are used for agent basic training before interacting with the environment. Moreover, the formal training is divided into four stages, in which DQR introduces a common framework to filter new interactive data for memory replay, including the process of classification, evaluation and sampling. The episodes are classified into different class based on opponent models and confrontation result. DQR can evaluate sample quality in each class and sample training batch with episode quality dynamically The new method is suitable for continuous action or discrete action space, and can makes pay-offs between specific demonstrations and exploration. Finally, we designed different initial engagement scenarios to train the agent models, whose basic algorithms are DQN, VPG, DDPG and SAC. In the test experiment, the winning rate of models can reach 0. 6 or more, and the best model SAC can reach 0. 8 for each stage, which means the feasibility and efficiency of the methods.

Details DOI

NeurIPS Conference 2022 Conference Paper

BadPrompt: Backdoor Attacks on Continuous Prompts

  • Xiangrui Cai
  • Haidong Xu
  • Sihan Xu
  • Ying Zhang
  • Yuan xiaojie

The prompt-based learning paradigm has gained much research attention recently. It has achieved state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios. While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based models. In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks. We observe that the few-shot scenarios have posed a great challenge to backdoor attacks on the prompt-based models, limiting the usability of existing NLP backdoor methods. To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts. Specially, BadPrompt first generates candidate triggers which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted labels. Then, it automatically selects the most effective and invisible trigger for each sample with an adaptive trigger optimization algorithm. We evaluate the performance of BadPrompt on five datasets and two continuous prompt models. The results exhibit the abilities of BadPrompt to effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baseline models by a large margin. The source code of BadPrompt is publicly available.

PDF Details

AIJ Journal 2022 Journal Article

Emotional conversation generation with heterogeneous graph neural network

  • Yunlong Liang
  • Fandong Meng
  • Ying Zhang
  • Yufeng Chen
  • Jinan Xu
  • Jie Zhou

The successful emotional conversation system depends on sufficient perception and appropriate expression of emotions. In a real-life conversation, humans firstly instinctively perceive emotions from multi-source information, including the emotion flow hidden in dialogue history, facial expressions, audio, and personalities of speakers. Then, they convey suitable emotions according to their personalities, but these multiple types of information are insufficiently exploited in emotional conversation fields. To address this issue, in this paper, we propose a heterogeneous graph-based model for emotional conversation generation. Firstly, we design a Heterogeneous Graph-Based Encoder to represent the conversation content (i. e. , the dialogue history, its emotion flow, facial expressions, audio, and speakers' personalities) with a heterogeneous graph neural network, and then predict suitable emotions for feedback. Secondly, we employ an Emotion-Personality-Aware Decoder to generate a response relevant to the conversation context as well as with appropriate emotions, through taking the encoded graph representations, the predicted emotions by the encoder and the personality of the current speaker as inputs. Experiments on both automatic and human evaluation show that our method can effectively perceive emotions from multi-source knowledge and generate a satisfactory response. Furthermore, based on the up-to-date text generator BART, our model still can achieve consistent improvement, which significantly outperforms some existing state-of-the-art models.

Details DOI

IJCAI Conference 2022 Conference Paper

Learning Unforgotten Domain-Invariant Representations for Online Unsupervised Domain Adaptation

  • Cheng Feng
  • Chaoliang Zhong
  • Jie Wang
  • Ying Zhang
  • Jun Sun
  • Yasuto Yokota

Existing unsupervised domain adaptation (UDA) studies focus on transferring knowledge in an offline manner. However, many tasks involve online requirements, especially in real-time systems. In this paper, we discuss Online UDA (OUDA) which assumes that the target samples are arriving sequentially as a small batch. OUDA tasks are challenging for prior UDA methods since online training suffers from catastrophic forgetting which leads to poor generalization. Intuitively, a good memory is a crucial factor in the success of OUDA. We formalize this intuition theoretically with a generalization bound where the OUDA target error can be bounded by the source error, the domain discrepancy distance, and a novel metric on forgetting in continuous online learning. Our theory illustrates the tradeoffs inherent in learning and remembering representations for OUDA. To minimize the proposed forgetting metric, we propose a novel source feature distillation (SFD) method which utilizes the source-only model as a teacher to guide the online training. In the experiment, we modify three UDA algorithms, i. e. , DANN, CDAN, and MCC, and evaluate their performance on OUDA tasks with real-world datasets. By applying SFD, the performance of all baselines is significantly improved.

PDF Details DOI

ICRA Conference 2022 Conference Paper

Traffic Context Aware Data Augmentation for Rare Object Detection in Autonomous Driving

  • Naifan Li
  • Fan Song
  • Ying Zhang
  • Pengpeng Liang
  • Erkang Cheng

Detection of rare objects (e. g. , traffic cones, traffic barrels and traffic warning triangles) is an important perception task to improve the safety of autonomous driving. Training of such models typically requires a large number of annotated data which is expensive and time consuming to obtain. To address the above problem, an emerging approach is to apply data augmentation to automatically generate cost-free training samples. In this work, we propose a systematic study on simple Copy-Paste data augmentation for rare object detection in autonomous driving. Specifically, local adaptive instance-level image transformation is introduced to generate realistic rare object masks from source domain to the target domain. Moreover, traffic scene context is utilized to guide the placement of masks of rare objects. To this end, our data augmentation generates training data with high quality and realistic characteristics by leveraging both local and global consistency. In addition, we build a new dataset named NM10k consisting 10k training images, 4k validation images and the corresponding labels with a diverse range of scenarios in autonomous driving. Experiments on NM10k show that our method achieves promising results on rare object detection. We also present a thorough study to illustrate the effectiveness of our local-adaptive and global constraints based Copy-Paste data augmentation for rare object detection. The data, development kit and more information of NM10k dataset are available online at: https://nullmax-vision.github.io.

Details

AAAI Conference 2021 Conference Paper

A Spatial Regulated Patch-Wise Approach for Cervical Dysplasia Diagnosis

  • Ying Zhang
  • Yifang Yin
  • Zhenguang Liu
  • Roger Zimmermann

Cervical dysplasia diagnosis via visual investigation is a challenging problem. Recent approaches use deep learning techniques to extract features and require the downsampling of high-resolution cervical screening images to smaller sizes for training. Such a reduction may result in the loss of visual details that appear weakly and locally within a cervical image. To overcome this challenge, our work divides an image into patches and then represents it from patch features. We aggregate patch patterns into an image feature in a weighted manner by considering the patch–image relationship. The weights are visualized as a heatmap to explain where the diagnosis results come from. We further introduce a spatial regulator to guide the classifier to focus on the cervix region and to adjust the weight distribution, without requiring any manual annotations of the cervix region. A novel iterative algorithm is designed to refine the regulator, which is able to capture the variations in cervix center locations and shapes. Experiments on an 18-year real-world dataset indicate a minimal of 3. 47%, 4. 59%, 8. 54% improvements over the state-of-the-art in accuracy, F1, and recall measures, respectively.

PDF Details

EAAI Journal 2021 Journal Article

BiS4EV: A fast routing algorithm considering charging stations and preferences for electric vehicles

  • Ying Zhang
  • Bin Wu
  • Yao-Yi Chiang
  • Xin Zhang
  • Yuanchang Chen
  • Muyang Li
  • Fanyu Li

Electric Vehicles (EVs) have grown in recent years as they have become a promising alternative to traditional fossil fuel-driven vehicles. As a result, new routing algorithms that consider both the locations of charging stations and the charging preferences of users are necessary to maintain urban traffic efficiency. This paper proposes a Constrained Route Planning (CRP) approach, called BiS4EV, to provide efficient route planning for EVs in large urban road networks. CRP is an NP-hard problem. BiS4EV introduces a two-layered structure to solve the CRP problem for EVs. In the first layer, BiS4EV integrates the locations of charging stations and the battery consumption on each road arc into a road network graph and proposes a fast routing algorithm that considers the constraints of battery capacity and charging preferences to find the shortest feasible path for EVs. The second layer incorporates charging policies to the obtained path, which forms a complete route plan for EVs. Such a two-layered method breaks down the complexity of the problem and avoids directly solving the entire NP-hard problem. We have proved the correctness of the proposed approach in theory, and the presented experimental results verified the efficiency and the effectiveness of BiS4EV from the practical aspect.

Details DOI

JMLR Journal 2021 Journal Article

Communication-Efficient Distributed Covariance Sketch, with Application to Distributed PCA

  • zengfeng Huang
  • Xuemin Lin
  • Wenjie Zhang
  • Ying Zhang

A sketch of a large data set captures vital properties of the original data while typically occupying much less space. In this paper, we consider the problem of computing a sketch of a massive data matrix $A\in\mathbb{R}^{n\times d}$ that is distributed across $s$ machines. Our goal is to output a matrix $B\in\mathbb{R}^{\ell\times d}$ which is significantly smaller than but still approximates $A$ well in terms of {covariance error}, i.e., $\|{A^TA-B^TB}\|_2$. Such a matrix $B$ is called a covariance sketch of $A$. We are mainly focused on minimizing the communication cost, which is arguably the most valuable resource in distributed computations. We show that there is a nontrivial gap between deterministic and randomized communication complexity for computing a covariance sketch. More specifically, we first prove an almost tight deterministic communication lower bound, then provide a new randomized algorithm with communication cost smaller than the deterministic lower bound. Based on a well-known connection between covariance sketch and approximate principle component analysis, we obtain better communication bounds for the distributed PCA problem. Moreover, we also give an improved distributed PCA algorithm for sparse input matrices, which uses our distributed sketching algorithm as a key building block. [abs] [ pdf ][ bib ] &copy JMLR 2021. ( edit, beta )

PDF Details

AAAI Conference 2021 Conference Paper

Enhanced Audio Tagging via Multi- to Single-Modal Teacher-Student Mutual Learning

  • Yifang Yin
  • Harsh Shrivastava
  • Ying Zhang
  • Zhenguang Liu
  • Rajiv Ratn Shah
  • Roger Zimmermann

Recognizing ongoing events based on acoustic clues has been a critical yet challenging problem that has attracted significant research attention in recent years. Joint audio-visual analysis can improve the event detection accuracy but may not always be feasible as under many circumstances only audio recordings are available in real-world scenarios. To solve the challenges, we present a novel visual-assisted teacherstudent mutual learning framework for robust sound event detection from audio recordings. Our model adopts a multimodal teacher network based on both acoustic and visual clues, and a single-modal student network based on acoustic clues only. Conventional teacher-student learning performs unsatisfactorily for knowledge transfer from a multi-modality network to a single-modality network. We thus present a mutual learning framework by introducing a single-modal transfer loss and a cross-modal transfer loss to collaboratively learn the audio-visual correlations between the two networks. Our proposed solution takes the advantages of joint audiovisual analysis in training while maximizing the feasibility of the model in use cases. Our extensive experiments on the DCASE17 and the DCASE18 sound event detection datasets show that our proposed method outperforms the state-of-theart audio tagging approaches.

PDF Details

AAAI Conference 2021 Conference Paper

Infusing Multi-Source Knowledge with Heterogeneous Graph Neural Network for Emotional Conversation Generation

  • Yunlong Liang
  • Fandong Meng
  • Ying Zhang
  • Yufeng Chen
  • Jinan Xu
  • Jie Zhou

The success of emotional conversation systems depends on sufficient perception and appropriate expression of emotions. In a real-world conversation, we firstly instinctively perceive emotions from multi-source information, including the emotion flow of dialogue history, facial expressions, and personalities of speakers, and then express suitable emotions according to our personalities, but these multiple types of information are insufficiently exploited in emotional conversation fields. To address this issue, we propose a heterogeneous graph-based model for emotional conversation generation. Specifically, we design a Heterogeneous Graph-Based Encoder to represent the conversation content (i. e. , the dialogue history, its emotion flow, facial expressions, and speakers’ personalities) with a heterogeneous graph neural network, and then predict suitable emotions for feedback. After that, we employ an Emotion-Personality-Aware Decoder to generate a response not only relevant to the conversation context but also with appropriate emotions, by taking the encoded graph representations, the predicted emotions from the encoder and the personality of the current speaker as inputs. Experimental results show that our model can effectively perceive emotions from multi-source knowledge and generate a satisfactory response, which significantly outperforms previous state-of-the-art models.

PDF Details

AAAI Conference 2021 Conference Paper

MTAAL: Multi-Task Adversarial Active Learning for Medical Named Entity Recognition and Normalization

  • Baohang Zhou
  • Xiangrui Cai
  • Ying Zhang
  • Wenya Guo
  • Xiaojie Yuan

Automated medical named entity recognition and normalization are fundamental for constructing knowledge graphs and building QA systems. When it comes to medical text, the annotation demands a foundation of expertise and professionalism. Existing methods utilize active learning to reduce costs in corpus annotation, as well as the multi-task learning strategy to model the correlations between different tasks. However, existing models do not take task-specific features for different tasks and diversity of query samples into account. To address these limitations, this paper proposes a multi-task adversarial active learning model for medical named entity recognition and normalization. In our model, the adversarial learning keeps the effectiveness of multi-task learning module and active learning module. The task discriminator eliminates the influence of irregular task-specific features. And the diversity discriminator exploits the heterogeneity between samples to meet the diversity constraint. The empirical results on two medical benchmarks demonstrate the effectiveness of our model against the existing methods.

PDF Details

YNIMG Journal 2021 Journal Article

Quantifying the variability of neural activation in working memory: A functional probabilistic atlas

  • Chen Chen
  • Ying Zhang
  • Zonglei Zhen
  • Yiying Song
  • Siyuan Hu
  • Jia Liu

Working memory is a fundamental cognitive ability that allows the maintenance and manipulation of information for a brief period of time. Previous studies found a set of brain regions activated during working memory tasks, such as the prefrontal and parietal cortex. However, little is known about the variability of neural activation in working memory. Here, we used functional magnetic resonance imaging to quantify individual, hemispheric, and sex differences of working memory activation in a large cohort of healthy adults (N = 477). We delineated subject-specific activated regions in each individual, including the frontal pole, middle frontal gyrus, frontal eye field, superior parietal lobule, insular, precuneus, and anterior cingulate cortex. A functional probabilistic atlas was created to quantify individual variability in working memory regions. More than 90% of the participants activated all seven regions in both hemispheres, but the intersection of regions across participants was markedly less (50%), indicating significant individual differences in working memory activations. Moreover, we found hemispheric and sex differences in activation location, extent, and magnitude. Most activation regions were larger in the right than in the left hemisphere, but the magnitude of activation did not follow a similar pattern. Men showed more extensive and stronger activations than women. Taken together, our functional probabilistic atlas quantified variabilities of neural activation in working memory, providing a robust spatial reference for standardization of functional localization.

Details DOI

AAAI Conference 2021 Conference Paper

Similarity Reasoning and Filtration for Image-Text Matching

  • Haiwen Diao
  • Ying Zhang
  • Lin Ma
  • Huchuan Lu

Image-text matching plays a critical role in bridging the vision and language, and great progress has been made by exploiting the global alignment between image and sentence, or local alignments between regions and words. However, how to make the most of these alignments to infer more accurate matching scores is still underexplored. In this paper, we propose a novel Similarity Graph Reasoning and Attention Filtration (SGRAF) network for image-text matching. Specifically, the vector-based similarity representations are firstly learned to characterize the local and global alignments in a more comprehensive manner, and then the Similarity Graph Reasoning (SGR) module relying on one graph convolutional neural network is introduced to infer relationaware similarities with both the local and global alignments. The Similarity Attention Filtration (SAF) module is further developed to integrate these alignments effectively by selectively attending on the significant and representative alignments and meanwhile casting aside the interferences of nonmeaningful alignments. We demonstrate the superiority of the proposed method with achieving state-of-the-art performances on the Flickr30K and MSCOCO datasets, and the good interpretability of SGR and SAF modules with extensive qualitative experiments and analyses.

PDF Details

JBHI Journal 2021 Journal Article

The Delineation of Fiducial Points for Non-Contact Radar Seismocardiogram Signals Without Concurrent ECG

  • Zongyang Xia
  • Md. Mobashir Hasan Shandhi
  • Yinghao Li
  • Omer T. Inan
  • Ying Zhang

Objective: Non-contact sensing of seismocardiogram (SCG) signals through a microwave Doppler radar is promising for biomedical applications. However, the delineation of fiducial points for radar SCG still relies on concurrent ECG which requires a contact sensor and limits the complete non-contact detection of SCG. Methods: Instead of ECG, a new reference signal, the radar displacement signal of heartbeat (RDH), was derived through the complex Fourier transform and the band pass filtering of the radar signal. The RDH signal was used to locate each cardiac cycle and mask the systolic profile, which was further used to detect an important fiducial point, aortic valve opening (AO). The beat-to-beat interval was estimated from AO-AO interval and compared with the gold standard, ECG R-to-R interval. Results: For the 22 subjects in the study, the evaluation of the AOs detected by RDH (AO RDH ) shows the average detection ratio can reach 90%, indicating a high ratio of the AO RDH that are exactly the same as AO detected using the ECG R-wave (AO ECG ). Additionally, the left ventricular ejection time (LVET) values estimated from the ensemble averaged radar waveform through AO RDH segmentation are within 2 ms of those through AO ECG segmentation, for all the detected subjects. Further analysis demonstrates that the beat-to-beat intervals calculated from AO RDH have an average root-mean-square-deviation (RMSD) of 53. 73 ms when compared with ECG R-to-R intervals, and have an average RMSD of 23. 47 ms after removing the beats in which AO cannot be identified. Conclusions: Radar signal RDH can be used as a reference signal to delineate fiducial points for non-contact radar SCG signals. Significance: This study can be applied to develop complete non-contact sensing of SCG and monitoring of vital signs, where contact-based SCG is not feasible.

Details DOI

IROS Conference 2020 Conference Paper

Cleaning Robot Operation Decision Based on Causal Reasoning and Attribute Learning *

  • Yapeng Li
  • Dongbo Zhang 0003
  • Feng Yin
  • Ying Zhang

In order to improve the operation ability of cleaning robots, this paper proposes a decision method for cleaning robot’s operation mode. Firstly, we use the hierarchical expression ability of deep network to obtain the attributes of garbage such as state, shape, distribution, size and so on. Then the causal relationship between the attributes and the operation modes can be built by using joint learning of association attributes with depth network model and causal inference. Based on this, a fuzzy inference decision network is designed. With the help of causal analysis, the structure of the decision model is greatly simplified. Compared with conventional fuzzy neural networks, the total parameters of the model are reduced by 2 / 3. The method proposed in this paper imitates the way that human dispose of different types of garbage and has good interpretability. The experimental results verify the effectiveness of the proposed method.

Details

IJCAI Conference 2020 Conference Paper

GoGNN: Graph of Graphs Neural Network for Predicting Structured Entity Interactions

  • Hanchen Wang
  • Defu Lian
  • Ying Zhang
  • Lu Qin
  • Xuemin Lin

Entity interaction prediction is essential in many important applications such as chemistry, biology, material science, and medical science. The problem becomes quite challenging when each entity is represented by a complex structure, namely structured entity, because two types of graphs are involved: local graphs for structured entities and a global graph to capture the interactions between structured entities. We observe that existing works on structured entity interaction prediction cannot properly exploit the unique graph of graphs model. In this paper, we propose a Graph of Graphs Neural Network, namely GoGNN, which extracts the features in both structured entity graphs and the entity interaction graph in a hierarchical way. We also propose the dual-attention mechanism that enables the model to preserve the neighbor importance in both levels of graphs. Extensive experiments on real-world datasets show that GoGNN outperforms the state-of-the-art methods on two representative structured entity interaction prediction tasks: chemical-chemical interaction prediction and drug-drug interaction prediction. Our code is available at Github.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Re-Attention for Visual Question Answering

  • Wenya Guo
  • Ying Zhang
  • Xiaoping Wu
  • Jufeng Yang
  • Xiangrui Cai
  • Xiaojie Yuan

Visual Question Answering (VQA) requires a simultaneous understanding of images and questions. Existing methods achieve well performance by focusing on both key objects in images and key words in questions. However, the answer also contains rich information which can help to better describe the image and generate more accurate attention maps. In this paper, to utilize the information in answer, we propose a reattention framework for the VQA task. We first associate image and question by calculating the similarity of each objectword pairs in the feature space. Then, based on the answer, the learned model re-attends the corresponding visual objects in images and reconstructs the initial attention map to produce consistent results. Benefiting from the re-attention procedure, the question can be better understood, and the satisfactory answer is generated. Extensive experiments on the benchmark dataset demonstrate the proposed method performs favorably against the state-of-the-art approaches.

PDF Details

IJCAI Conference 2020 Conference Paper

Risk Guarantee Prediction in Networked-Loans

  • Dawei Cheng
  • Xiaoyang Wang
  • Ying Zhang
  • Liqing Zhang

The guaranteed loan is a debt obligation promise that if one corporation gets trapped in risks, its guarantors will back the loan. When more and more companies involve, they subsequently form complex networks. Detecting and predicting risk guarantee in these networked-loans is important for the loan issuer. Therefore, in this paper, we propose a dynamic graph-based attention neural network for risk guarantee relationship prediction (DGANN). In particular, each guarantee is represented as an edge in dynamic loan networks, while companies are denoted as nodes. We present an attention-based graph neural network to encode the edges that preserve the financial status as well as network structures. The experimental result shows that DGANN could significantly improve the risk prediction accuracy in both the precision and recall compared with state-of-the-art baselines. We also conduct empirical studies to uncover the risk guarantee patterns from the learned attentional network features. The result provides an alternative way for loan risk management, which may inspire more work in the future.

PDF Details DOI

YNIMG Journal 2019 Journal Article

Enhanced high-frequency precuneus-cortical effective connectivity is associated with decreased sensory gating following total sleep deprivation

  • Liwei Zhang
  • Yue Huang
  • Ying Zhang
  • Wei Xin
  • Yongcong Shao
  • Yebing Yang

Sleep deprivation decreases an individual's cognitive function. When cognitive impairment reaches a certain level, human errors occur and may even result in accidents. Previous research has shown that sensory gating is a basic mechanism in cognitive function, but only limited studies have so far reported how it is affected by sleep deprivation. This study aimed to analyze the effects of sleep deprivation on sensory gating and its cognitive and neural mechanisms. Thirty-six healthy subjects participated in our study. The resting-state, auditory P50-task electroencephalography (EEG) recordings and the psychomotor vigilance task (PVT) were performed at resting wakefulness (RW) and after 36 h of total sleep deprivation (TSD). Changes in P50 suppression before and after sleep deprivation were recorded, and the isolated effective coherence (iCoh) was employed for analyzing effective connectivity based on EEG data during the resting-state and P50 tasks. Subjects demonstrated reduced P50 suppression and prolonged PVT reaction time after TSD compared with RW. Effective connectivity analysis of resting-state EEG data showed that sleep deprivation decreased the connectivity from the right middle occipital gyrus (RMOG)/Rcuneus to left inferior/middle temporal gyrus (LITG/LMTG) and left parahippocampal/fusiform gyrus (LPH/LFG). EEG data analysis during the P50 task showed that, in addition to the aforementioned connectivity changes, the directed high-frequency effective connectivity from the left precuneus to the left superior/middle frontal gyrus (LSFG/LMFG), LITG/LMTG, LPH/LFG, and left middle occipital gyrus (LMOG)/Lcuneus increased. P50 suppression in Cz positively correlated with PVT reaction time. This study reveals that the precuneus is a key brain region in neural network correlates of sensory gating, and that changes in its effective connectivity with other regions (including LSFG/LMFG, LPH/LFG, LMOG/LCuneus, and LITG/LMTG) are important for decreasing sensory gating after TSD.

Details DOI

IJCAI Conference 2019 Conference Paper

E²GAN: End-to-End Generative Adversarial Network for Multivariate Time Series Imputation

  • Yonghong Luo
  • Ying Zhang
  • Xiangrui Cai
  • Xiaojie Yuan

The missing values, appear in most of multivariate time series, prevent advanced analysis of multivariate time series data. Existing imputation approaches try to deal with missing values by deletion, statistical imputation, machine learning based imputation and generative imputation. However, these methods are either incapable of dealing with temporal information or multi-stage. This paper proposes an end-to-end generative model E²GAN to impute missing values in multivariate time series. With the help of the discriminative loss and the squared error loss, E²GAN can impute the incomplete time series by the nearest generated complete time series at one stage. Experiments on multiple real-world datasets show that our model outperforms the baselines on the imputation accuracy and achieves state-of-the-art classification/regression results on the downstream applications. Additionally, our method also gains better time efficiency than multi-stage method on the training of neural networks.

PDF Details

YNIMG Journal 2019 Journal Article

Uncovering cortical activations of discourse comprehension and their overlaps with common large-scale neural networks

  • XiaoHong Yang
  • Huijie Li
  • Nan Lin
  • XiuPing Zhang
  • YinShan Wang
  • Ying Zhang
  • Qian Zhang
  • XiNian Zuo

We conducted a meta-analysis of 78 task-based functional magnetic resonance imaging (fMRI) studies (1976 total participants) to reveal underlying brain activations and their overlap with large-scale neural networks in the brain during general discourse comprehension and its sub-processes. We found that discourse comprehension involved a neural system consisting of widely distributed brain regions that comprised not only the bilateral perisylvian language zones, but also regions in the superior and medial frontal cortex and the medial temporal lobe. Moreover, this neural system can be categorized into several sub-systems representing various sub-processes of discourse comprehension, with the left inferior frontal gyrus and middle temporal gyrus serving as core regions across all sub-processes. At a large-scale network level, we found that discourse comprehension relied most heavily on the default network, particularly on its dorsal medial subsystem. The pattern associated with large-scale network cooperation varied according to the respective sub-processes required. Our results reveal the functional dissociation within the discourse comprehension neural system and highlight the flexible involvements of large-scale networks.

Details DOI

IJCAI Conference 2018 Conference Paper

Medical Concept Embedding with Time-Aware Attention

  • Xiangrui Cai
  • Jinyang Gao
  • Kee Yuan Ngiam
  • Beng Chin Ooi
  • Ying Zhang
  • Xiaojie Yuan

Embeddings of medical concepts such as medication, procedure and diagnosis codes in Electronic Medical Records (EMRs) are central to healthcare analytics. Previous work on medical concept embedding takes medical concepts and EMRs as words and documents respectively. Nevertheless, such models miss out the temporal nature of EMR data. On the one hand, two consecutive medical concepts do not indicate they are temporally close, but the correlations between them can be revealed by the time gap. On the other hand, the temporal scopes of medical concepts often vary greatly (e. g. , common cold and diabetes). In this paper, we propose to incorporate the temporal information to embed medical codes. Based on the Continuous Bag-of-Words model, we employ the attention mechanism to learn a ``soft'' time-aware context window for each medical concept. Experiments on public and proprietary datasets through clustering and nearest neighbour search tasks demonstrate the effectiveness of our model, showing that it outperforms five state-of-the-art baselines.

PDF Details

NeurIPS Conference 2018 Conference Paper

Multivariate Time Series Imputation with Generative Adversarial Networks

  • Yonghong Luo
  • Xiangrui Cai
  • Ying Zhang
  • Jun Xu
  • Yuan xiaojie

Multivariate time series usually contain a large number of missing values, which hinders the application of advanced analysis methods on multivariate time series data. Conventional approaches to addressing the challenge of missing values, including mean/zero imputation, case deletion, and matrix factorization-based imputation, are all incapable of modeling the temporal dependencies and the nature of complex distribution in multivariate time series. In this paper, we treat the problem of missing value imputation as data generation. Inspired by the success of Generative Adversarial Networks (GAN) in image generation, we propose to learn the overall distribution of a multivariate time series dataset with GAN, which is further used to generate the missing values for each sample. Different from the image data, the time series data are usually incomplete due to the nature of data recording process. A modified Gate Recurrent Unit is employed in GAN to model the temporal irregularity of the incomplete time series. Experiments on two multivariate time series datasets show that the proposed model outperformed the baselines in terms of accuracy of imputation. Experimental results also showed that a simple model on the imputed data can achieve state-of-the-art results on the prediction tasks, demonstrating the benefits of our model in downstream applications.

PDF Details

IJCAI Conference 2018 Conference Paper

Text Emotion Distribution Learning via Multi-Task Convolutional Neural Network

  • Yuxiang Zhang
  • Jiamei Fu
  • Dongyu She
  • Ying Zhang
  • Senzhang Wang
  • Jufeng Yang

Emotion analysis of on-line user generated textual content is important for natural language processing and social media analytics tasks. Most of previous emotion analysis approaches focus on identifying users’ emotional states from text by classifying emotions into one of the finite categories, e. g. , joy, surprise, anger and fear. However, there exists ambiguity characteristic for the emotion analysis, since a single sentence can evoke multiple emotions with different intensities. To address this problem, we introduce emotion distribution learning and propose a multi-task convolutional neural network for text emotion analysis. The end-to-end framework optimizes the distribution prediction and classification tasks simultaneously, which is able to learn robust representations for the distribution dataset with annotations of different voters. While most work adopt the majority voting scheme for the ground truth labeling, we also propose a lexiconbased strategy to generate distributions from a single label, which provides prior information for the emotion classification. Experiments conducted on five public text datasets (i. e. , SemEval, Fairy Tales, ISEAR, TEC, CBET) demonstrate that our proposed method performs favorably against the state-of-the-art approaches.

PDF Details

AAAI Conference 2017 Conference Paper

Finding Critical Users for Social Network Engagement: The Collapsed k-Core Problem

  • Fan Zhang
  • Ying Zhang
  • Lu Qin
  • Wenjie Zhang
  • Xuemin Lin

In social networks, the leave of critical users may significantly break network engagement, i. e. , lead a large number of other users to drop out. A popular model to measure social network engagement is k-core, the maximal induced subgraph in which every vertex has at least k neighbors. To identify critical users for social network engagement, we propose the collapsed kcore problem: given a graph G, a positive integer k and a budget b, we aim to find b vertices in G such that the deletion of the b vertices leads to the smallest k-core. We prove the problem is NP-hard. Then, an efficient algorithm is proposed, which significantly reduces the number of candidate vertices to speed up the computation. Our comprehensive experiments on 9 real-life social networks demonstrate the effectiveness and efficiency of our proposed method.

PDF Details

NeurIPS Conference 2016 Conference Paper

On Multiplicative Integration with Recurrent Neural Networks

  • Yuhuai Wu
  • Saizheng Zhang
  • Ying Zhang
  • Yoshua Bengio
  • Russ Salakhutdinov

We introduce a general simple structural design called “Multiplicative Integration” (MI) to improve recurrent neural networks (RNNs). MI changes the way of how the information flow gets integrated in the computational building block of an RNN, while introducing almost no extra parameters. The new structure can be easily embedded into many popular RNN models, including LSTMs and GRUs. We empirically analyze its learning behaviour and conduct evaluations on several tasks using different RNN models. Our experimental results demonstrate that Multiplicative Integration can provide a substantial performance boost over many of the existing RNN models.

PDF Details

NeurIPS Conference 2016 Conference Paper

Professor Forcing: A New Algorithm for Training Recurrent Networks

  • Alex Lamb
  • Anirudh Goyal ALIAS PARTH GOYAL
  • Ying Zhang
  • Saizheng Zhang
  • Aaron Courville
  • Yoshua Bengio

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network’s own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

PDF Details

ICAPS Conference 2011 Conference Paper

Online Planning for a Material Control System for Liquid Crystal Display Manufacturing

  • Minh Binh Do
  • Kazumichi Okajima
  • Serdar Uckun
  • Fumio Hasegawa
  • Yukihiro Kawano
  • Koji Tanaka
  • Lara S. Crawford
  • Ying Zhang

The hyper-modular printer control project at PARC has proven that a tightly integrated model-based planning and control framework can effectively control a complex physical system. Recently, we have successfully applied this framework to another application: planning for the Material Control System (MCS) of Liquid Crystal Display (LCD) manufacturing plant in a joint project between the Embedded Reasoning Area at PARC and the Products Development Center at the IHI Corporation. The model-based planner created at PARC was able to successfully solve a diverse set of test scenarios provided by IHI, including those that were deemed very difficult by the IHI experts. The short projecttime (2 months) proved that model-based planning is a flexible framework that can adapt quickly to novel applications. In this paper, we will introduce this complex domain and describe the adaptation process of the Plantrol online planner. The main contributions are: (1) introducing a successful application of general-purpose planning; (2) outline the timeline-based online temporal planner; and (3) description of a complex warehouse management problem that can serve as an attractive benchmark domain for planning.

Details

ICRA Conference 2007 Conference Paper

Mobile Sensor Networks Self Localization based on Multi-dimensional Scaling

  • Changhua Wu
  • Weihua Sheng
  • Ying Zhang

In this paper, we define a mobile self-localization (MSL) problem for sparse mobile sensor networks, and propose an algorithm named mobility assisted MDS-MAP(P), based on multi-dimensional scaling (MDS) for solving the problem. For sparse sensor networks, all the existing localization algorithms fail to work properly due to the lack of distance or connectivity data to uniquely calculate the geo-locations. In MSL, we use mobile sensors to add extra distance constraints to a sparse network, by moving the mobile sensors in the area of deployment and recording distances to neighbors at some intermediate locations. MSL can also be used for localizing and tracking mobile objects in a robotic or body sensor network. Experiments and evaluations of the new algorithm are provided.

Details

ICRA Conference 2004 Conference Paper

PolyBot and PolyKinetic™ System: a Modular Robotic Platform for Education

  • Alex Golovinsky
  • Mark Yim
  • Ying Zhang
  • Craig Eldershaw
  • Dave Duff

Modular robotics has been an active area of research for the last decade. In this paper, we argue that it also provides an excellent platform for education. PolyBot is a type of modular reconfigurable robot developed at PARC. The PolyKinetic/spl trade/ System provides a robotic scripting language and programming environment for controlling this type of robot. Based on experience with mentoring high school students, and running a tutorial for robotic researchers at IROS'03, we show that, PolyBot and the PolyKinetic/spl trade/ System is effective for educational activities at multiple levels. These include playing, exploring, building, programming, competing, and most of all, learning while having fun.

Details

IROS Conference 2003 Conference Paper

Motion planning with narrow C-space passages

  • Craig Eldershaw
  • Mark Yim
  • Ying Zhang
  • Kimon Roufas
  • Dave Duff

A common point of weakness in many path/motion planners is dealing with configuration spaces (C-spaces) that involve narrow gaps of passages. This is caused by the indirect representation of the C-space inherent in those planners. Unfortunately many real environments and tasks give rise to such situations. While a small number of planners do exist which reliably work in these environments, they in turn have problems when the robot has many controllable degrees of freedom. High degree of freedom motion in confined spaces is a typical problem encountered by PARC's PolyBot platform. The algorithm put forward in this paper is designed to work directly with the C-space obstacles' geometry, and so is not hampered by narrow passages. The planner is a homotopic one which solves general problems in n-dimensional C-space. The algorithm's theoretical characteristics are shown to compare very favourably with other contending planners in these specialised domains. Results of some preliminary testing are given.

Details

IROS Conference 2003 Conference Paper

Phase automata: a programming model of locomotion gaits for scalable chain-type modular robots

  • Ying Zhang
  • Mark Yim
  • Craig Eldershaw
  • Dave Duff
  • Kimon Roufas

Modular reconfigurable robots have the potential for great versatility and robustness; however, programming locomotion gaits for hundreds of modules remains a challenge. In this paper we present a formal model for programming locomotion gaits in chain-type modular robots: phase automata. A phase automation is an event-driven state automation with an initial phase delay. The phase delay is normally a real value between 0 and 1. Phase automata are compact representation of locomotion gaits and capable of being embedded and distributed across modules. The concepts of phase automata have been implemented on both PCs and embedded micro-processors. An XML script language and programming interface for phase automata are being built. Locomotion gaits programmed using phase automata have been tested both in simulation with 100+ modules and in hardware with 50+ modules.

Details

IROS Conference 2002 Conference Paper

A general constraint-based control framework with examples in modular self-reconfigurable robots

  • Ying Zhang
  • Markus P. J. Fromherz
  • Lara S. Crawford
  • Yi Shang

In this paper, we advocate a general constraint-based control framework that is highly promising for building control systems with complex dynamic structures, such as modular self-reconfigurable robots. In this framework, a controller consists of constraint solving components distributed in a network of embedded processors. Constraint solvers are goal oriented deliberative agents that can be used as control regulators or as information retrievers. The framework is built on the attribute/service model (ASM), a middleware for coordinating actuators, sensors and tasks in distributed real-time embedded systems. The communications and coordination among the services are realized via shared attributes. Examples of controlling a modular self-reconfigurable robot are illustrated in the paper.

Details

IROS Conference 2001 Conference Paper

Closed-chain motion with large mechanical advantage

  • Mark Yim
  • Dave Duff
  • Ying Zhang

One of the constraints that severely limit the capability of highly redundant manipulator arms is the actuator torque limits. This paper presents a way to achieve large effective forces from weak actuators by exploiting the large mechanical advantage that results from systems near singularities. While large mechanical advantages have been applied near singularities in many instances, this method allows the application of this large force over a large distance. It is applied specifically to closed chain mechanisms and demonstrated on the PolyBot modular self-reconfigurable robot.

Details

IROS Conference 2001 Conference Paper

Software architecture for modular self-reconfigurable robots

  • Ying Zhang
  • Kimon Roufas
  • Mark Yim

Modular, self-reconfigurable robots show the promise of great versatility, robustness and low cost. However, programming such robots for specific tasks, with hundreds of modules and each of which with multiple actuators and sensors, can be tedious and error-prone. The extreme versatility of the modular systems requires a new paradigm in programming. We present a software architecture for this type of robot, in particular the PolyBot, which has been developed through its third generation. The architecture, based on the properties of the PolyBot electro-mechanical design, features a multi-master/multi-slave structure in a multi-threaded environment, with three layers of communication protocols. The architecture is currently being implemented for Motorola PowerPC using vxWorks.

Details

TCS Journal 1995 Journal Article

Constraint nets: a semantic model for hybrid dynamic systems

  • Ying Zhang
  • Alan K. Mackworth

Hybrid dynamic systems are systems consisting of a nontrivial mixture of discrete and continuous components, such as a controller realized by a combination of digital and analog circuits, a robot composed of a digital controller and a physical plant, or a robotic system consisting of a computer-controlled robot coupled to a continuous environment. Hybrid dynamic systems are more general then traditional real-time systems. The former can be composed of continuous subsystems in addition to discrete and event-controlled components. In this paper, we develop a semantic model, constraint nets (CN), for hybrid systems. CN captures the most general structure of dynamic systems so that systems with discrete and continuous time, discrete and continuous variables, and asynchronous as well as synchronous event structures, can be modeled in a unitary framework. Using aggregation operators, a system can be modeled hierarchically in CN; therefore, the dynamics of the environment as well as the dynamics of the plant and the dynamics of the controller can be modeled individually and then integrated. Based on abstract algebra and topology, CN supports multiple levels of abstraction, so that a system can be analyzed at different levels of detail. CN also provides a rigorous formal programming semantics for the design of hybrid real-time embedded systems.

Details DOI
v2026.09.13