Author name cluster

Liu Yang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

33 papers

2 author rows

AAAI Conference 2026 Conference Paper

FedDNA: DNA Sequence Reconstruction via Deep Evidential Learning and Personalized Federated Aggregation

Haiyan Lin
Qi Shen
Fei Zhu
Zixuan Qin
Liu Yang
Yuping Duan

DNA-based data storage offers an attractive alternative to traditional media due to its exceptional density, durability, and sustainability. However, errors introduced across the DNA storage pipeline critically impede accurate sequence reconstruction from noisy sequencing reads. This paper addresses the DNA sequence reconstruction problem by proposing FedDNA, a novel Personalized Federated Learning (PFL) framework based on Evidential Deep Learning (DEL), designed for DNA storage environments. FedDNA quantifies robust predictive uncertainty through a novel evidence fusion mechanism that aggregates evidence from each noisy read in a cluster, thereby enhancing client-level prediction reliability. For efficient sequence modeling and reconstruction from these noisy clusters, its architecture employs a convolution-enhanced Mamba encoder and an LSTM decoder. To address prohibitive centralized training costs, privacy concerns, and data heterogeneity across diverse DNA storage data, FedDNA integrates PFL and designs an innovative uncertainty-driven personalized aggregation strategy based on epistemic and aleatoric decomposition, for which we also provide rigorous theoretical generalization bounds. Experimental results demonstrate FedDNA achieves superior reconstruction performance on DNA storage data with heterogeneity, highlighting its potential for secure and efficient DNA storage systems.

PDF Details DOI

AAAI Conference 2026 Conference Paper

GIER: Addressing Class Imbalance in GNNs Through Experience Replay

Liu Yang
Chuyao Liu
Zidong Wang
Tingxuan Chen
Mengni Chen
Hongyu Zhang

The prevalent class imbalance in real-world graphs significantly affects the performance of Graph Neural Networks (GNNs). Existing methods for analyzing graph imbalance ignore the influence of minority nodes during the dynamic model training process, resulting in performance limitations. In this paper, we focus on minority class information during model training, identifying and defining the minority class forgetting phenomenon that exists in graph imbalanced method training processes. To address this issue, we propose Graph Imbalance Experience Replay(GIER) framework. On one hand, the method enhances the model's ability to mine minority node information in historical data, thereby achieving feature completion for minority class nodes. On the other hand, the proposed short-term confidence mechanism allows the model to adaptively calibrate the topological relationships in high-confidence nodes, thereby mitigating the model's tendency to propagate erroneous information about minority classes during training. GIER is a unified framework consisting of two synergistic components: Long-term Subgraph Memory (LSM) constructs multi-period feature-representative subgraphs to address distribution imbalance, and Short-term Confidence Calibration (SCC) dynamically reconstructs graph topology through degree-aware node selection and confidence-based filtering to address topological imbalance. The extensive experimental results demonstrate that GIER effectively improves the classification performance of GNNs on imbalanced graphs, achieving up to a 3.44% improvement in BAcc over the state-of-the-art, and is particularly effective in extreme scenarios with very small minority classes.

PDF Details DOI

AAAI Conference 2026 Conference Paper

HiFC-GAN: Hierarchical Feature-Constrained GAN for Optical-to-SAR Transfer in SAR Target Classification

Hao Zheng
Meiguang Zheng
Zhigang Hu
Liu Yang
Aikun Xu
Tingxuan Chen
Rongchang Zhao
Boyu Wang

The limited availability of high-quality training data poses a persistent challenge for synthetic aperture radar (SAR) target classification. Existing data augmentation methods mainly adopt a simplistic application of GAN-based style transfer techniques to directly synthesize pseudo-SAR images from optical images. However, our in-depth analysis of this cross-modal conversion reveals that such straightforward strategies primarily focus on transferring high-level semantic information (e.g., target shapes), thus failing to adequately capture the essential low-level features unique to SAR imagery (e.g., scattering textures). To address this inherent trade-off between high-level semantic preservation and low-level feature authenticity, we propose a Hierarchical Feature-Constrained GAN (HiFC-GAN) tailored for optical-to-SAR style transfer. Specifically, HiFC-GAN enhances the representation of low-level SAR features by introducing local texture contrast constraints at shallow layers, while introducing explicit feature mapping constraints at deeper layers to maintain high-level semantic consistency throughout the reconstruction process. Experimental results demonstrate that HiFC-GAN significantly outperforms existing GAN-based techniques in image generation quality, particularly improving the low-level feature authenticity of pseudo-SAR images. Moreover, the generated pseudo-SAR images further improve the performance of downstream target classification tasks, yielding accuracy gains ranging from 3.56% to 5.90% on average with mainstream CNN-based models.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Intra-Class Unbiased Prototype Aggregation and Classifier Collaboration for Personalized Federated Learning

Hao Zheng
Shiyu Song
Zhigang Hu
Meiguang Zheng
Liu Yang
Aikun Xu
Rongchang Zhao
Ruizhi Pu

Prototype-based personalized federated learning methods have emerged as a promising strategy due to their ability to represent client-specific class characteristics effectively through learned class prototypes. These prototypes capture salient features of client-local data, facilitating personalized model adaptation. However, existing prototype-based aggregation strategies predominantly rely on weighted averaging, implicitly assuming prototype consistency across clients. This assumption neglects the intrinsic heterogeneity and non-independent and identically distributed (non-IID) nature of client data, compelling diverse local prototypes to align toward a singular global prototype and consequently causing significant aggregation bias. Motivated by observations from intra-class feature saliency analysis, we identify that clients inherently emphasize distinct feature regions even for the same class. To leverage this intra-class diversity, we introduce FedIC, a novel prototype clustering and collaborative classifier optimization approach. Specifically, FedIC first clusters prototypes based on intra-class similarity to form intra-class prototype subspaces, ensuring that aggregation occurs exclusively within each cluster, thus eliminating the bias stemming from forced global unification. To further exploit the benefits of intra-cluster collaboration, we quantify the combined predictive gains of classifiers from clients within the same cluster as a function of classifier combination weights. This targeted aggregation and collaborative optimization strategy effectively circumvents the bias introduced by global alignment. Extensive experiments under various non-IID settings show that FedIC significantly outperforms existing Prototype-based and Clustered PFL Methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Less Is More: Rethinking Parameter-Efficient Fine-Tuning from a Subtractive Perspective

Tianqi Jiang
Liu Yang
Xi-Le Zhao
Zixuan Qin
Qinghua Hu

Currently, pretrained models are rapidly scaling in size, which substantially increases the cost of fine-tuning them for downstream tasks. To address this challenge, parameter-efficient fine-tuning (PEFT) methods have been developed to optimize a minimal set of parameters for adaptation. While current PEFT approaches predominantly employ an "additive'' strategy, introducing learnable modules into inputs or architectures, neglect the inherent knowledge embedded within pretrained models, which may be redundant or even conflict with downstream tasks. This limitation leads to increased inference latency and suboptimal transfer performance, particularly in scenarios with significant domain gaps. In this paper, we propose a Subtractive Fine-tuning Paradigm(SFP), which converts multiple redundant operations within the original module into a linear transformation to enhance inference speed and model performance. Specifically, we introduce a compact filter block to replace specific module with interference and redundancy in the original structure to reduce model conflicts. By using a pseudo inverse matrix to construct filter block, ensuring that it can inherit the knowledge of the replacement module, and then freezing the rest of the model, only fine-tuning the filter block is performed to eliminate interference and redundant knowledge, thereby enhancing the model’s adaptability to downstream tasks. Experimental results demonstrate that our SFP outperforms existing PEFT methods in accuracy while decreasing the overall model parameters by 12%. Compared to full fine-tuning, the accuracy has increased by 8.47%(74.04% vs. 65.57%, VTAB).

PDF Details DOI

EAAI Journal 2026 Journal Article

Robust synchronization of chaotic systems using noise-resistant gradient neural dynamics: Design and application

Guancheng Wang
Liu Yang
Fenghao Zhuang
Lingbo Han
Zhihao Hao
Xiuchun Xiao
Cong Lin

The synchronization of chaotic systems has found extensive applications in various fields, including secure communication, financial modeling, and image encryption. However, in practical scenarios, chaotic system dynamics are often significantly affected by external noise, which degrades trajectory stability and adversely impacts synchronization performance. Addressing noise-induced degradation has therefore become a critical research focus in chaotic system studies. As a key area of artificial intelligence, neural dynamics (ND) plays a significant role in modeling and optimizing complex systems. In this paper, a robust controller is designed to address this issue within a general master–slave chaotic system framework by employing Noise-Resistant Gradient Neural Dynamics (NRGND), which effectively mitigates the effects of noise and enhances synchronization efficacy. Notably, the controller requires only that the master and slave systems share the same dimensionality and that their system states and corresponding time derivatives are observable. In addition, theoretical analyses of the controller under various noises are conducted, demonstrating its exceptional convergence and robustness. Next, experiments conducted in representative chaotic systems under various noisy scenarios illustrate the superior performance of the NRGND-based controller in achieving synchronization. Finally, the performance of the NRGND controller was evaluated in two applications. The proposed method achieves image encryption and decryption by driving the Lorenz and Lu chaotic systems into synchronization. Furthermore, the NRGND-based robotic motion control scheme demonstrated its robustness and stability under noisy conditions, highlighting its potential for real-world engineering applications.

Details DOI

TMLR Journal 2026 Journal Article

VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction

Yadi Cao
Yuxuan Liu
Liu Yang
Rose Yu
Hayden Schaeffer
Stanley Osher

In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, existing ICONs process each spatial point as an individual token, severely limiting computational efficiency when handling dense data in higher spatial dimensions. We propose \textit{Vision In-Context Operator Networks} (VICON), which integrate vision transformer architectures to efficiently process 2D data through patch-wise operations while preserving ICON's adaptability to multi-physics systems and varying timesteps. Evaluated across three fluid dynamics benchmarks, VICON significantly outperforms state-of-the-art baselines DPOT and MPP, reducing the average last-step rollout error by 37.9\% compared to DPOT and 44.7\% compared to MPP, while requiring only 72.5\% and 34.8\% of their respective inference times. VICON naturally supports flexible rollout strategies with varying timestep strides, enabling immediate deployment in \textit{imperfect measurement systems} where sampling frequencies may differ or frames might be dropped—common challenges in real-world settings—without requiring retraining or interpolation. In these realistic scenarios, VICON exhibits remarkable robustness, experiencing only 24.41\% relative performance degradation compared to 71.37\%-74.49\% degradation in baseline methods, demonstrating its versatility for deployment in realistic applications. Our scripts for processing datasets and code are publicly available at https://github.com/Eydcao/VICON.

PDF Details

IROS Conference 2025 Conference Paper

A Bio-inspired Spherical Soft Magnetic Millirobot for Gastrointestinal Applications

Yulin Li
Zhaorui Hong
Yuhao Zhao
Shuohao Zhang
Xin Zhao
Liu Yang

Gastroscopy and colonoscopy have become the fundamental tools for gastrointestinal (GI) tract diagnosis and treatment. Conventional tethered devices usually lead to the use of anesthetic agents and patient discomfort. Capsule endoscopy is becoming an ideal alternative, however, the smooth capsule shape may hinder its active locomotion and retention in the GI tract. Here, we propose a spherical soft magnetic millirobot (S 2 M 2 robot) that integrates a virus-like spherical body with protrusions and an octopus-inspired sucker design for enhanced physical capabilities. The protruding suckers (radius: 2. 2-4. 8 mm) on its surface enable efficient locomotion performance (maximum angular velocity: 8 r/s, maximum speed: 180 mm/s) with strong adhesive ability (maximum force: 3. 5 N). The ex vivo experiments in a swine stomach demonstrate the robot’s motion on the slippery surface of the gastric mucosa and the effectiveness of the pressure-based drug delivery system. The in vitro and ex vivo results highlight the superior mobility and controllability, showcasing its potential as a carrier robot for the next-generation capsule endoscopy.

Details

IJCAI Conference 2025 Conference Paper

ADFormer: Aggregation Differential Transformer for Passenger Demand Forecasting

Haichen Wang
Liu Yang
Xinyuan Zhang
Haomin Yu
Ming Li
Jilin Hu

Passenger demand forecasting helps optimize vehicle scheduling, thereby improving urban efficiency. Recently, attention-based methods have been used to adequately capture the dynamic nature of spatio-temporal data. However, existing methods that rely on heuristic masking strategies cannot fully adapt to the complex spatio-temporal correlations, hindering the model from focusing on the right context. These works also overlook the high-level correlations that exist in the real world. Effectively integrating these high-level correlations with the original correlations is crucial. To fill this gap, we propose the Aggregation Differential Transformer (ADFormer), which offers new insights to demand forecasting promotion. Specifically, we utilize Differential Attention to capture the original spatial correlations and achieve attention denoising. Meanwhile, we design distinct aggregation strategies based on the nature of space and time. Then, the original correlations are unified with the high-level correlations, enabling the model to capture holistic spatio-temporal relations. Experiments conducted on taxi and bike datasets confirm the effectiveness and efficiency of our model, demonstrating its practical value. The code is available at https: //github. com/decisionintelligence/ADFormer.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

An Out-Of-Distribution Membership Inference Attack Approach for Cross-Domain Graph Attacks

Jinyan Wang
Liu Yang
Yuecen Wei
Jiaxuan Si
Chenhao Guo
Qingyun Sun
Xianxian Li
Xingcheng Fu

Graph Neural Network-based methods face privacy leakage risks due to the introduction of topological structures about the targets, which allows attackers to bypass the target's prior knowledge of the sensitive attributes and realize membership inference attacks (MIA) by observing and analyzing the topology distribution. As privacy concerns grow, the assumption of MIA, which presumes that attackers can obtain an auxiliary dataset with the same distribution, is increasingly deviating from reality. In this paper, we categorize the distribution diversity issue in real-world MIA scenarios as an Out-Of-Distribution (OOD) problem, and propose a novel Graph OOD Membership Inference Attack (GOOD-MIA) to achieve cross-domain graph attacks. Specifically, we construct shadow subgraphs with distributions from different domains to model the diversity of real-world data. We then explore the stable node representations that remain unchanged under external influences and consider eliminating redundant information from confounding environments and extracting task-relevant key information to more clearly distinguish between the characteristics of training data and unseen data. This OOD-based design makes cross-domain graph attacks possible. Finally, we perform risk extrapolation to optimize the attack's domain adaptability during attack inference to generalize the attack to other domains. Experimental results demonstrate that GOOD-MIA achieves superior attack performance in datasets designed for multiple domains.

PDF Details DOI

AAAI Conference 2025 Conference Paper

ConFREE: Conflict-free Client Update Aggregation for Personalized Federated Learning

Hao Zheng
Zhigang Hu
Liu Yang
Meiguang Zheng
Aikun Xu
Boyu Wang

Negative transfer (NF) is a critical challenge in personalized federated learning (pFL). Existing methods primarily focus on adapting local data distribution on the client side, which can only resist NF, rather than avoid NF itself. To tackle NF at its root, we investigate its mechanism through the lens of the global model, and argue that it is caused by update conflicts among clients during server aggregation. In light of this, we propose a conflict-free client update aggregation strategy (ConFREE), which enables us to avoid NF in pFL. Specifically, ConFREE guides the global update direction by constructing a conflict-free guidance vector through projection and utilizes the optimal local improvements of the worst-performing clients near the guidance vector to regularize server aggregation. This prevents the conflicting components of updates from transferring, achieving balanced updates across different clients. Notably, ConFREE is model-agnostic and can be straightforwardly adopted as a complement to enhance various existing NF-resistance methods implemented on the client side. Extensive experiments demonstrate substantial improvements to existing pFL algorithms by leveraging ConFREE.

PDF Details DOI

AAAI Conference 2025 Conference Paper

HLMEA: Unsupervised Entity Alignment Based on Hybrid Language Models

Xiongnan Jin
Zhilin Wang
Jinpeng Chen
Liu Yang
Byungkook Oh
Seung-won Hwang
Jianqiang Li

Entity alignment (EA) is crucial for integrating knowledge graphs (KGs) constructed from diverse sources. Conventional unsupervised EA approaches attempt to eliminate human intervention but often suffer from accuracy limitations. With the rise of large language models (LLMs), leveraging their capabilities for EA presents a promising direction. However, it introduces new challenges: formulating the LLM-based EA problem and extracting the background knowledge in LLMs to realize EA without human intervention. This paper proposes HLMEA, a novel hybrid language model-based unsupervised EA method. HLMEA formulates the EA task into a filtering and single-choice problem and synergistically integrates small language models (SLMs) and LLMs. Specifically, SLMs filter candidate entities based on textual representations generated from KG triples. Then, LLMs refine this selection to identify the most semantically aligned entities. An iterative self-training mechanism allows SLMs to distill knowledge from LLM outputs, enhancing the EA ability of hybrid language models in subsequent rounds cooperatively. We also conducted extensive experiments on benchmark datasets to evaluate HLMEA's performance. The results demonstrate that HLMEA significantly outperforms unsupervised and even supervised EA baselines, proving its potential for scalable and effective EA across large KGs. The code and data are available at \url{https://github.com/xnjin-ai/HLMEA}.

PDF Details DOI

TMLR Journal 2025 Journal Article

Preference Discerning with LLM-Enhanced Generative Retrieval

Fabian Paischer
Liu Yang
Linfeng Liu
Shuai Shao
Kaveh Hassani
Jiacheng Li
Ricky T. Q. Chen
Zhang Gabriel Li

In sequential recommendation, models recommend items based on user's interaction history. To this end, current models usually incorporate information such as item descriptions and user intent or preferences. User preferences are usually not explicitly given in open-source datasets, and thus need to be approximated, for example via large language models (LLMs). Current approaches leverage approximated user preferences only during training and rely solely on the past interaction history for recommendations, limiting their ability to dynamically adapt to changing preferences, potentially reinforcing echo chambers. To address this issue, we propose a new paradigm, namely *preference discerning*, which explicitly conditions a generative recommendation model on user preferences in natural language within its context. To evaluate *preference discerning*, we introduce a novel benchmark that provides a holistic evaluation across various scenarios, including preference steering and sentiment following. Upon evaluating current state-of-the-art methods on our benchmark, we discover that their ability to dynamically adapt to evolving user preferences is limited. To address this, we propose a new method named Mender (**M**ultimodal Prefer**en**ce **D**iscern**er**), which achieves state-of-the-art performance in our benchmark. Our results show that Mender effectively adapts its recommendation guided by human preferences, even if not observed during training, paving the way toward more flexible recommendation models.

PDF Details

TMLR Journal 2025 Journal Article

Unifying Generative and Dense Retrieval for Sequential Recommendation

Liu Yang
Fabian Paischer
Kaveh Hassani
Jiacheng Li
Shuai Shao
Zhang Gabriel Li
Yun He
Xue Feng

Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user through inner product computation between the user and all item representations. While effective, these approaches incur high memory and computational costs due to the need to store and compare a unique embedding for each item--leading to lower resource efficiency. In contrast, the recently proposed generative retrieval paradigm offers a promising alternative by directly predicting item indices using a generative model trained on semantic IDs that encapsulate items’ semantic information. Despite its potential for large-scale applications, a comprehensive comparison between generative retrieval and sequential dense retrieval under fair conditions is still lacking, leaving open questions regarding performance and resource efficiency trade-offs. To address this, we compare these two approaches under controlled conditions on academic benchmarks and observe performance gaps, with dense retrieval showing stronger ranking performance, while generative retrieval provides greater resource efficiency. Motivated by these observations, we propose LIGER (LeveragIng dense retrieval for GEnerative Retrieval), a hybrid model that combines the strengths of these two widely used approaches. LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences between the two methods, and enhancing cold-start item recommendation in the evaluated datasets. This hybrid approach provides insight into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks.

PDF Details

IJCAI Conference 2023 Conference Paper

Globally Consistent Federated Graph Autoencoder for Non-IID Graphs

Kun Guo
Yutong Fang
Qingqing Huang
Yuting Liang
Ziyao Zhang
Wenyu He
Liu Yang
Kai Chen

Graph neural networks (GNNs) have been applied successfully in many machine learning tasks due to their advantages in utilizing neighboring information. Recently, with the global enactment of privacy protection regulations, federated GNNs have gained increasing attention in academia and industry. However, the graphs owned by different participants could be non-independently-and-identically distributed (non-IID), leading to the deterioration of federated GNNs' accuracy. In this paper, we propose a globally consistent federated graph autoencoder (GCFGAE) to overcome the non-IID problem in unsupervised federated graph learning via three innovations. First, by integrating federated learning with split learning, we train a unique global model instead of FedAvg-styled global and local models, yielding results consistent with that of the centralized GAE. Second, we design a collaborative computation mechanism considering overlapping vertices to reduce communication overhead during forward propagation. Third, we develop a layer-wise and block-wise gradient computation strategy to reduce the space and communication complexity during backward propagation. Experiments on real-world datasets demonstrate that GCFGAE achieves not only higher accuracy but also around 500 times lower communication overhead and 1000 times smaller space overhead than existing federated GNN models.

PDF Details DOI

EAAI Journal 2023 Journal Article

Hybrid VMD-CNN-GRU-based model for short-term forecasting of wind power considering spatio-temporal features

Zeni Zhao
Sining Yun
Lingyun Jia
Jiaxin Guo
Yao Meng
Ning He
Xuejuan Li
Jiarong Shi

Accurate and reliable short-term forecasting of wind power is vital for balancing energy and integrating wind power into a grid. A novel hybrid deep learning model is designed in this study to increase the prediction accuracy of short-term wind power forecasting on a wind farm in Jiang County, Shanxi, China. The proposed hybrid deep learning model comprises variable mode decomposition (VMD), convolutional neural network (CNN), and gated recurrent unit (GRU). VMD substantially reduces the volatility of wind speed sequences. CNN automatically extracts complex spatial features from wind power data, and GRU can directly extract temporal features from historical input data. The forecasting accuracy of the combined VMD-CNN-GRU model is higher than that of any single model for wind power. The study used data obtained in 15 min intervals from the wind farm to determine the effectiveness of the proposed model against other advanced models. Compared with the other deep learning models, VMD-CNN-GRU is the best at short-term forecasting, with an RMSE of 1. 5651, MAE of 0. 8161, MAPE of 11. 62%, and R2 of 0. 9964. This method is valuable for practical applications and can be used to maintain safe wind farm operations in the future.

Details DOI

IROS Conference 2022 Conference Paper

Downwash-aware Control Allocation for Over-actuated UAV Platforms

Yao Su 0001
Chi Chu
Meng Wang 0051
Jiarui Li
Liu Yang
Yixin Zhu 0001
Hangxin Liu

Tracking position and orientation independently affords more agile maneuver for over-actuated multirotor Unmanned Aerial Vehicles (UAVs) while introducing undesired downwash effects; downwash flows generated by thrust generators may counteract others due to close proximity, which significantly threatens the stability of the platform. The complexity of modeling aerodynamic airflow challenges control algorithms from properly compensating for such a side effect. Leveraging the input redundancies in over-actuated UAVs, we tackle this issue with a novel control allocation framework that considers downwash effects and explores the entire allocation space for an optimal solution. This optimal solution avoids downwash effects while providing high thrust efficiency within the hardware constraints. To the best of our knowledge, ours is the first formal derivation to investigate the downwash effects on over-actuated UAVs. We verify our framework on different hardware configurations in both simulation and experiment.

Details

TIST Journal 2022 Journal Article

Improving Availability of Vertical Federated Learning: Relaxing Inference on Non-overlapping Data

Zhenghang Ren
Liu Yang
Kai Chen

Vertical Federated Learning (VFL) enables multiple parties to collaboratively train a machine learning model over vertically distributed datasets without data privacy leakage. However, there is a limitation of the current VFL solutions: current VFL models fail to conduct inference on non-overlapping samples during inference. This limitation seriously damages the VFL model’s availability because, in practice, overlapping samples may only take up a small portion of the whole data at each party which means a large part of inference tasks will fail. In this article, we propose a novel VFL framework which enables federated inference on non-overlapping data. Our framework regards the distributed features as privileged information which is available in the training period but disappears during inference. We distill the knowledge of such privileged features and transfer them to the parties’ local model which only processes local features. Furthermore, we adopt Oblivious Transfer (OT) to preserve data ID privacy during training and inference. Empirically, we evaluate the model on the real-world dataset collected from Criteo and Taobao. Besides, we also provide a security analysis of the proposed framework.

Details DOI

NeurIPS Conference 2022 Conference Paper

Rare Gems: Finding Lottery Tickets at Initialization

Kartik Sreenivasan
Jy-yong Sohn
Liu Yang
Matthew Grinde
Alliot Nagle
Hongyi Wang
Eric Xing
Kangwook Lee

Large neural networks can be pruned to a small fraction of their original size, with little loss in accuracy, by following a time-consuming "train, prune, re-train" approach. Frankle & Carbin conjecture that we can avoid this by training lottery tickets, i. e. , special sparse subnetworks found at initialization, that can be trained to high accuracy. However, a subsequent line of work presents concrete evidence that current algorithms for finding trainable networks at initialization, fail simple baseline comparisons, e. g. , against training random sparse subnetworks. Finding lottery tickets that train to better accuracy compared to simple baselines remains an open problem. In this work, we resolve this open problem by proposing Gem-Miner which finds lottery tickets at initialization that beat current baselines. Gem-Miner finds lottery tickets trainable to accuracy competitive or better than Iterative Magnitude Pruning (IMP), and does so up to $19\times$ faster.

PDF Details

AAAI Conference 2021 Conference Paper

Flow-based Generative Models for Learning Manifold to Manifold Mappings

Xingjian Zhen
Rudrasis Chakraborty
Liu Yang
Vikas Singh

Many measurements or observations in computer vision and machine learning manifest as non-Euclidean data. While recent proposals (like spherical CNN) have extended a number of deep neural network architectures to manifold-valued data, and this has often provided strong improvements in performance, the literature on generative models for manifold data is quite sparse. Partly due to this gap, there are also no modality transfer/translation models for manifold-valued data whereas numerous such methods based on generative models are available for natural images. This paper addresses this gap, motivated by a need in brain imaging – in doing so, we expand the operating range of certain generative models (as well as generative models for modality transfer) from natural images to images with manifold-valued measurements. Our main result is the design of a two-stream version of GLOW (flow-based invertible generative models) that can synthesize information of a field of one type of manifold-valued measurements given another. On the theoretical side, we introduce three kinds of invertible layers for manifold-valued data, which are not only analogous to their functionality in flow-based generative models (e. g. , GLOW) but also preserve the key benefits (determinants of the Jacobian are easy to calculate). For experiments, on a large dataset from the Human Connectome Project (HCP), we show promising results where we can reliably and accurately reconstruct brain images of a field of orientation distribution functions (ODF) from diffusion tensor images (DTI), where the latter has a 5× faster acquisition time but at the expense of worse angular resolution.

PDF Details

ICLR Conference 2021 Conference Paper

Long Range Arena: A Benchmark for Efficient Transformers

Yi Tay
Mostafa Dehghani 0001
Samira Abnar
Yikang Shen
Dara Bahri
Philip Pham
Jinfeng Rao
Liu Yang

Transformers do not scale very well to long sequence lengths largely because of quadratic self-attention complexity. In the recent months, a wide spectrum of efficient, fast Transformers have been proposed to tackle this problem, more often than not claiming superior or comparable model quality to vanilla Transformer models. To this date, there is no well-established consensus on how to evaluate this class of models. Moreover, inconsistent benchmarking on a wide spectrum of tasks and datasets makes it difficult to assess relative model quality amongst many models. This paper proposes a systematic and unified benchmark, Long Range Arena, specifically focused on evaluating model quality under long-context scenarios. Our benchmark is a suite of tasks consisting of sequences ranging from $1K$ to $16K$ tokens, encompassing a wide range of data types and modalities such as text, natural, synthetic images, and mathematical expressions requiring similarity, structural, and visual-spatial reasoning. We systematically evaluate ten well-established long-range Transformer models (Reformers, Linformers, Linear Transformers, Sinkhorn Transformers, Performers, Synthesizers, Sparse Transformers, and Longformers) on our newly proposed benchmark suite. Long Range Arena paves the way towards better understanding this class of efficient Transformer models, facilitates more research in this direction, and presents new challenging tasks to tackle.

Details

SODA Conference 2020 Conference Paper

Computing and Testing Small Connectivity in Near-Linear Time and Queries via Fast Local Cut Algorithms

Sebastian Forster
Danupon Nanongkai
Liu Yang
Thatchaphol Saranurak
Sorrachai Yingchareonthawornchai

Consider the following “local” cut-detection problem in a directed graph: We are given a seed vertex x and need to remove at most k edges so that at most v edges can be reached from x (a “local” cut) or output ⊥ to indicate that no such cut exists. If we are given query access to the input graph, then this problem can in principle be solved without reading the whole graph and with query complexity depending on k and ν. In this paper we consider a slack variant of this problem where, when such a cut exists, we can output a cut with up to O ( kν ) edges reachable from x. We present a simple randomized algorithm spending O ( k 2 ν ) time and O ( kν ) queries for the above variant, improving in particular a previous time bound of O ( k O ( k ) ν ) by Chechik et al. [SODA’17]. We also extend our algorithm to handle an approximate variant. We demonstrate that these local algorithms are versatile primitives for designing substantially improved algorithms for classic graph problems by providing the following three applications. (Throughout, Õ ( T ) hides polylog( T ).) A randomized algorithm for the classic k -vertex connectivity problem that takes near-linear time when k = O (polylog( n )), namely Õ ( m + nk 3 ) time in undirected graphs. Prior to our work, the state of the art for this range of k were linear-time algorithms for k ≤ 3 [Tarjan FOCS’71; Hopcroft, Tarjan SICOMP’73] and a recent algorithm with Õ ( m + n 4/3 k 7/3 ) time [Nanongkai et al. , STOC’19]. The story is the same for directed graphs where our Õ (mk 2 )-time algorithm is near-linear when k = O (polylog( n )). Our techniques also yield an improved approximation scheme. Property testing algorithms for k -edge and -vertex connectivity with query complexities that are near-linear in k, exponentially improving the state-of-the-art. This resolves two open problems, one by Goldreich and Ron [STOC’97] and one by Orenstein and Ron [Theor. Comput. Sci. ’11]. A faster algorithm for computing the maximal k- edge connected subgraphs, improving prior work of Chechik et al. [SODA’17].

Details

ICML Conference 2020 Conference Paper

Sparse Sinkhorn Attention

Yi Tay
Dara Bahri
Liu Yang
Donald Metzler
Da-Cheng Juan

We propose Sparse Sinkhorn Attention, a new efficient and sparse method for learning to attend. Our method is based on differentiable sorting of internal representations. Concretely, we introduce a meta sorting network that learns to generate latent permutations over sequences. Given sorted sequences, we are then able to compute quasi-global attention with only local windows, improving the memory efficiency of the attention module. To this end, we propose new algorithmic innovations such as Causal Sinkhorn Balancing and SortCut, a dynamic sequence truncation method for tailoring Sinkhorn Attention for encoding and/or decoding purposes. Via extensive experiments on algorithmic seq2seq sorting, language modeling, pixel-wise image generation, document classification and natural language inference, we demonstrate that our memory efficient Sinkhorn Attention method is competitive with vanilla attention and consistently outperforms recently proposed efficient Transformer models such as Sparse Transformers.

Details

FLAP Journal 2019 Journal Article

Finite Degradation Structures.

Antoine Rauzy
Liu Yang

Probabilistic risk and safety analyses are used in virtually all industries to assess whether the risk of operating complex technical systems is low enough to be socially acceptable. As of today, these analyses rely mainly on stochastic Boolean models such as fault trees or reliability block diagrams. These models are coarse approximations of the behavior of the systems under study. In this article, we introduce the notion of finite degradation structure. Finite degradation structures encode the degradation order among the states of multistate systems, i.e. models in which variables can take a finite number of values rather than just two. This extension of Boolean formalisms makes it possible to increase significantly the capacity of expression without increasing significantly the complexity of the calculation of risk indicators. Technically, finite degradation structures are finite semi-lattices associated with a random process. They form a monoidal category and provide a unified algebraic framework for Boolean reliability models and multistate systems. They shed a new light on central notions of system reliability theory such as those of coherent models and minimal cutsets.

Details

IJCAI Conference 2018 Conference Paper

Beyond Similar and Dissimilar Relations: A Kernel Regression Formulation for Metric Learning

Pengfei Zhu
Ren Qi
Qinghua Hu
Qilong Wang
Changqing Zhang
Liu Yang

Most existing metric learning methods focus on learning a similarity or distance measure relying on similar and dissimilar relations between sample pairs. However, pairs of samples cannot be simply identified as similar or dissimilar in many real-world applications, e. g. , multi-label learning, label distribution learning or tasks with continuous decision values. To this end, in this paper we propose a novel relation alignment metric learning (RAML) formulation to handle the metric learning problem in those scenarios. Since the relation of two samples can be measured by the difference degree of the decision values, motivated by the consistency of the sample relations in the feature space and decision space, our proposed RAML utilizes the sample relations in the decision space to guide the metric learning in the feature space. Specifically, our RAML method formulates metric learning as a kernel regression problem, which can be efficiently optimized by the standard regression solvers. We carry out several experiments on the single-label classification, multi-label classification, and label distribution learning tasks, to demonstrate that our method achieves favorable performance against the state-of-the-art methods.

PDF Details

TCS Journal 2018 Journal Article

Bounds on the minimax rate for estimating a prior over a VC class from independent learning tasks

Liu Yang
Steve Hanneke
Jaime Carbonell

We study the optimal rates of convergence for estimating a prior distribution over a VC class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior. We specifically derive upper and lower bounds on the optimal rates under a smoothness condition on the correct prior, with the number of samples per data set equal the VC dimension. These results have implications for the improvements achievable via transfer learning. We additionally extend this setting to real-valued function, where we establish consistency of an estimator for the prior, and discuss an additional application to a preference elicitation problem in algorithmic economics.

Details DOI

TCS Journal 2018 Journal Article

Testing piecewise functions

Steve Hanneke
Liu Yang

This work explores the query complexity of property testing for general piecewise functions on the real line, in the active and passive property testing settings. The results are proven under an abstract zero-measure crossings condition, which has as special cases piecewise constant functions and piecewise polynomial functions. We find that, in the active testing setting, the query complexity of testing general piecewise functions is independent of the number of pieces. We also identify the optimal dependence on the number of pieces in the query complexity of passive testing in the special case of piecewise constant functions.

Details DOI

JMLR Journal 2015 Journal Article

Minimax Analysis of Active Learning

Steve Hanneke
Liu Yang

This work establishes distribution-free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes, under various noise models. The results reveal a number of surprising facts. In particular, under the noise model of Tsybakov (2004), the minimax label complexity of active learning with a VC class is always asymptotically smaller than that of passive learning, and is typically significantly smaller than the best previously-published upper bounds in the active learning literature. In high-noise regimes, it turns out that all active learning problems of a given VC dimension have roughly the same minimax label complexity, which contrasts with well-known results for bounded noise. In low-noise regimes, we find that the label complexity is well-characterized by a simple combinatorial complexity measure we call the star number. Interestingly, we find that almost all of the complexity measures previously explored in the active learning literature have worst-case values exactly equal to the star number. We also propose new active learning strategies that nearly achieve these minimax label complexities. [abs] [ pdf ][ bib ] &copy JMLR 2015. ( edit, beta )

PDF Details

IJCAI Conference 2015 Conference Paper

Sparse Probabilistic Matrix Factorization by Laplace Distribution for Collaborative Filtering

Liping Jing
Peng Wang
Liu Yang

In recommendation systems, probabilistic matrix factorization (PMF) is a state-of-the-art collaborative filtering method by determining the latent features to represent users and items. However, two major issues limiting the usefulness of PMF are the sparsity problem and long-tail distribution. Sparsity refers to the situation that the observed rating data are sparse, which results in that only part of latent features are informative for describing each item/user. Long tail distribution implies that a large fraction of items have few ratings. In this work, we propose a sparse probabilistic matrix factorization method (SPMF) by utilizing a Laplacian distribution to model the item/user factor vector. Laplacian distribution has ability to generate sparse coding, which is beneficial for SPMF to distinguish the relevant and irrelevant latent features with respect to each item/user. Meanwhile, the tails in Laplacian distribution are comparatively heavy, which is rewarding for SPMF to recommend the tail items. Furthermore, a distributed Gibbs sampling algorithm is developed to efficiently train the proposed sparse probabilistic model. A series of experiments on Netflix and Movielens datasets have been conducted to demonstrate that SPMF outperforms the existing PMF and its extended version Bayesian PMF (BPMF), especially for the recommendation of tail items.

PDF Details

NeurIPS Conference 2013 Conference Paper

Buy-in-Bulk Active Learning

Liu Yang
Jaime Carbonell

In many practical applications of active learning, it is more cost-effective to request labels in large batches, rather than one-at-a-time. This is because the cost of labeling a large batch of examples at once is often sublinear in the number of examples in the batch. In this work, we study the label complexity of active learning algorithms that request labels in a given number of batches, as well as the tradeoff between the total number of queries and the number of rounds allowed. We additionally study the total cost sufficient for learning, for an abstract notion of the cost of requesting the labels of a given number of examples at once. In particular, we find that for sublinear cost functions, it is often desirable to request labels in large batches (i. e. , buying in bulk); although this may increase the total number of labels requested, it reduces the total cost required for learning.

PDF Details

NeurIPS Conference 2011 Conference Paper

Active Learning with a Drifting Distribution

Liu Yang

We study the problem of active learning in a stream-based setting, allowing the distribution of the examples to change over time. We prove upper bounds on the number of prediction mistakes and number of label requests for established disagreement-based active learning algorithms, both in the realizable case and under Tsybakov noise. We further prove minimax lower bounds for this problem.

PDF Details

NeurIPS Conference 2008 Conference Paper

Semi-supervised Learning with Weakly-Related Unlabeled Data : Towards Better Text Categorization

Liu Yang
Rong Jin
Rahul Sukthankar

The cluster assumption is exploited by most semi-supervised learning (SSL) methods. However, if the unlabeled data is merely weakly related to the target classes, it becomes questionable whether driving the decision boundary to the low density regions of the unlabeled data will help the classification. In such case, the cluster assumption may not be valid; and consequently how to leverage this type of unlabeled data to enhance the classification accuracy becomes a challenge. We introduce Semi-supervised Learning with Weakly-Related Unlabeled Data" (SSLW), an inductive method that builds upon the maximum-margin approach, towards a better usage of weakly-related unlabeled information. Although the SSLW could improve a wide range of classification tasks, in this paper, we focus on text categorization with a small training pool. The key assumption behind this work is that, even with different topics, the word usage patterns across different corpora tends to be consistent. To this end, SSLW estimates the optimal word-correlation matrix that is consistent with both the co-occurrence information derived from the weakly-related unlabeled documents and the labeled documents. For empirical evaluation, we present a direct comparison with a number of state-of-the-art methods for inductive semi-supervised learning and text categorization; and we show that SSLW results in a significant improvement in categorization accuracy, equipped with a small training set and an unlabeled resource that is weakly related to the test beds. "

PDF Details

AAAI Conference 2006 Conference Paper

An Efficient Algorithm for Local Distance Metric Learning

Liu Yang
Rahul Sukthankar

Learning application-specific distance metrics from labeled data is critical for both statistical classification and information retrieval. Most of the earlier work in this area has focused on finding metrics that simultaneously optimize compactness and separability in a global sense. Specifically, such distance metrics attempt to keep all of the data points in each class close together while ensuring that data points from different classes are separated. However, particularly when classes exhibit multimodal data distributions, these goals conflict and thus cannot be simultaneously satisfied. This paper proposes a Local Distance Metric (LDM) that aims to optimize local compactness and local separability. We present an efficient algorithm that employs eigenvector analysis and bound optimization to learn the LDM from training data in a probabilistic framework. We demonstrate that LDM achieves significant improvements in both classification and retrieval accuracy compared to global distance learning and kernel-based KNN.

PDF Details