Author name cluster

Xiaoguang Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

1 author row

AAAI Conference 2025 Conference Paper

Genomics Data Lossless Compression with (S, K)-Mer Encoding and Deep Neural Networks

Hui Sun
Liping Yi
Huidong Ma
Yongxia Sun
Yingfeng Zheng
Wenwen Cui
Meng Yan
Gang Wang

Learning-based compression shows competitive compression ratios for genomics data. It often includes three types of compressors: static, adaptive and semi-adaptive. However, these existing compressors suffer from inferior compression ratios or throughput, and adaptive compressors also faces model cold-start problems. To address these issues, we propose DeepGeCo, a novel genomics data lossless adaptive compression framework with (s,k)-mer encoding and deep neural networks, involving three compression modes (MINI for static, PLUS for adaptive, ULTRA for semi-adaptive) for flexible requirements of compression ratios or throughput. In DeepGeCo, (1) we develop BiGRU and Transformer as the backbone to build Warm-Start and Supporter models in terms of cold-start problems. (2) We introduce (s,k)-mer encoding to pre-process genomics data before feeding it into the DNN model for improve model throughput, and we propose a new metric - Ranking of Throughput and Compression Ratio (RTCR) for effective encoding parameters selection. (3) We design a threshold controller and a probabilistic mixer within the backbone to balance compression ratios and model throughput. Experiments on 10 real-world datasets show that DeepGeCo's three compression modes improve up to a 22.949X average throughput and up to a 31.095% average compression ratio improvement while occupying low CPU or GPU memory.

PDF Details DOI

AAAI Conference 2025 Conference Paper

pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning

Liping Yi
Han Yu
Chao Ren
Gang Wang
Xiaoguang Liu
Xiaoxiao Li

Federated learning (FL), as a privacy-preserving collaborative machine learning paradigm, has attracted significant interest from industry and academia. To allow each data owner (FL client) to train a heterogeneous and personalized local model based on its local data distribution, system resources and requirements on model structure, the field of model-heterogeneous personalized federated learning (MHPFL) has emerged. Existing MHPFL approaches either rely on the availability of a public dataset with special characteristics to facilitate knowledge transfer, incur high computational and communication costs, or face potential model leakage risks. To address these limitations, we propose a model-heterogeneous personalized Federated learning approach based on generalized proxy feature Extractor Sharing (pFedES) for supervised image classification tasks. (1) We devise a shared small proxy homogeneous feature extractor before each client's heterogeneous local model. (2) Clients train them via the proposed iterative learning to enable the exchange of global generalized knowledge and local personalized knowledge. (3) The small proxy local homogeneous extractors produced after local training are uploaded to the server for aggregation to facilitate knowledge fusion across clients. We theoretically prove pFedES converges with a non-convex convergence rate O(1/T). Experiments on 3 benchmark datasets against 9 baselines demonstrate that pFedES performs state-of-the-art model accuracy while maintaining efficient communication and computation.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Federated Model Heterogeneous Matryoshka Representation Learning

Liping Yi
Han Yu
Chao Ren
Gang Wang
Xiaoguang Liu
Xiaoxiao Li

Model heterogeneous federated learning (MHeteroFL) enables FL clients to collaboratively train models with heterogeneous structures in a distributed fashion. However, existing MHeteroFL methods rely on training loss to transfer knowledge between the client model and the server model, resulting in limited knowledge exchange. To address this limitation, we propose the **Fed**erated model heterogeneous **M**atryoshka **R**epresentation **L**earning (**FedMRL**) approach for supervised learning tasks. It adds an auxiliary small homogeneous model shared by clients with heterogeneous local models. (1) The generalized and personalized representations extracted by the two models' feature extractors are fused by a personalized lightweight representation projector. This step enables representation fusion to adapt to local data distribution. (2) The fused representation is then used to construct Matryoshka representations with multi-dimensional and multi-granular embedded representations learned by the global homogeneous model header and the local heterogeneous model header. This step facilitates multi-perspective representation learning and improves model learning capability. Theoretical analysis shows that FedMRL achieves a $O(1/T)$ non-convex convergence rate. Extensive experiments on benchmark datasets demonstrate its superior model accuracy with low communication and computational costs compared to seven state-of-the-art baselines. It achieves up to 8. 48% and 24. 94% accuracy improvement compared with the state-of-the-art and the best same-category baseline, respectively.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

Liping Yi
Han Yu
Zhuan Shi
Gang Wang
Xiaoguang Liu
Lizhen Cui
Xiaoxiao Li

Federated learning (FL) is a privacy-preserving collaboratively machine learning paradigm. Traditional FL requires all data owners (a. k. a. FL clients) to train the same local model. This design is not well-suited for scenarios involving data and/or system heterogeneity. Model-Heterogeneous Personalized FL (MHPFL) has emerged to address this challenge. Existing MHPFL approaches often rely on a public dataset with the same nature as the learning task, or incur high computation and communication costs. To address these limitations, we propose the Federated Semantic Similarity Aggregation (FedSSA) approach for supervised classification tasks, which splits each client's model into a heterogeneous (structure-different) feature extractor and a homogeneous (structure-same) classification header. It performs local-to-global knowledge transfer via semantic similarity-based header parameter aggregation. In addition, global-to-local knowledge transfer is achieved via an adaptive parameter stabilization strategy which fuses the seen-class parameters of historical local headers with that of the latest global header for each client. FedSSA does not rely on public datasets, while only requiring partial header parameter transmission to save costs. Theoretical analysis proves the convergence of FedSSA. Extensive experiments present that FedSSA achieves up to 3. 62% higher accuracy, 15. 54 times higher communication efficiency, and 15. 52 times higher computational efficiency compared to 7 state-of-the-art MHPFL baselines.

PDF Details DOI

EAAI Journal 2023 Journal Article

Meta pseudo labels for anomaly detection via partially observed anomalies

Sinong Zhao
Zhaoyang Yu
Sifan Li
Xiaofei Wang
Trent G. Marbach
Gang Wang
Xiaoguang Liu

General anomaly detection has been an important research due to its broad and significant applications. Those algorithms that are based on weakly supervised or partially observed anomalies have attracted particular interest currently. However, most such algorithms treat the unlabeled set as a substitute for normal samples and ignore the potential anomalies in it, which on the one hand causes the noise of the normal set, and on the other hand, fails make full use of the abnormal supervision information. To address this issue, we propose a meta-pseudo-label based framework for anomaly detection (MPAD). The framework strives to obtain effective pseudo anomalies from the unlabeled samples to supplement the observed anomaly set. Specifically, a teacher network is improved based on the feedback of a student network on a validation set, thereby generating more conducive pseudo anomalies to assist the student network while incurring less confirmation bias. In addition, we also design a model roll-back scheme to guarantee the performance of the student network. Extensive experiments show that the proposed MPAD algorithm outperforms current popular algorithms on five real datasets, and the framework can effectively improve the utilization of unlabeled samples while maintaining an advanced degree of robustness. In addition, the results of ablation experiments confirm the effectiveness of each module of our MPAD. Parameter experiments provide sufficient reference for parameter selecting.

Details DOI

AIJ Journal 2020 Journal Article

PopMNet: Generating structured pop music melodies using neural networks

Jian Wu
Xiaoguang Liu
Xiaolin Hu
Jun Zhu

Recently, many deep learning models have been proposed to generate symbolic melodies. However, generating pop music melodies with well organized structures remains to be challenging. In this paper, we present a melody structure-based model called PopMNet to generate structured pop music melodies. The melody structure is defined by pairwise relations, specifically, repetition and sequence, between all bars in a melody. PopMNet consists of a Convolutional Neural Network (CNN)-based Structure Generation Net (SGN) and a Recurrent Neural Network (RNN)-based Melody Generation Net (MGN). The former generates melody structures and the latter generates melodies conditioned on the structures and chord progressions. The proposed model is compared with four existing models AttentionRNN, LookbackRNN, MidiNet and Music Transformer. The results indicate that the melodies generated by our model contain much clearer structures compared to those generated by other models, as confirmed by human behavior experiments.

Details DOI

AAMAS Conference 2018 Conference Paper

Slim-DP: A Multi-Agent System for Communication-Efficient Distributed Deep Learning

Shizhao Sun
Wei Chen
Jiang Bian
Xiaoguang Liu
Tie-Yan Liu

To afford the huge computational cost, large-scale deep neural networks (DNN) are usually trained on the distributed system, especially the widely-used parameter server architecture, consisting of a parameter server as well as multiple local workers with powerful GPU cards. During the training, local workers frequently pull the global model and push their computed gradients from/to the parameter server. Due to the limited bandwidth, such frequent communication will cause severe bottleneck for the training acceleration. As recent attempts to address this problem, quantization methods have been proposed to compress the gradients for efficient communication. However, such methods overlook the effects of compression on the model performance such that they either suffer from a low compression ratio or an accuracy drop. In this paper, to better address this problem, we investigate the distributed deep learning as a multi-agent system (MAS) problem. Specifically, 1) local workers and the parameter server are separate agents in the system; 2) the objective of these agents is to maximize the efficacy of the learned model through their cooperative interactions; 3) the strategy of the agents describes how they take actions, i. e. communicate their computed gradients or the global model; 4) rational agents always select the best-response strategy with the optimal utility. Inspired by this, we design a MAS approach for distributed training of DNN. In our method, the agents first estimate the utility (i. e. , the benefit to help improve the model) of each action (i. e. , transferring a subset of the gradients or the global model), and then take the best-response strategy based on their estimated utilities mixed with ϵ-random exploration. We call our new method Slim-DP as it, being different from the standard data-parallelism, only communicates a subset of the gradient or the global model. Our experimental results demonstrate that our proposed Slim-DP can reduce more communication cost and achieve better speedup without loss of accuracy than the standard data parallelism and its quantization version. ∗This work was done when the author was visiting Microsoft Research Asia. Proc. of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018), M. Dastani, G. Sukthankar, E. André, S. Koenig (eds.), July 10–15, 2018, Stockholm, Sweden. © 2018 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). All rights reserved.

PDF

AAAI Conference 2016 Conference Paper

On the Depth of Deep Neural Networks: A Theoretical View

Shizhao Sun
Wei Chen
Liwei Wang
Xiaoguang Liu
Tie-Yan Liu

People believe that depth plays an important role in success of deep neural networks (DNN). However, this belief lacks solid theoretical justiﬁcations as far as we know. We investigate role of depth from perspective of margin bound. In margin bound, expected error is upper bounded by empirical margin error plus Rademacher Average (RA) based capacity term. First, we derive an upper bound for RA of DNN, and show that it increases with increasing depth. This indicates negative impact of depth on test performance. Second, we show that deeper networks tend to have larger representation power (measured by Betti numbers based complexity) than shallower networks in multi-class setting, and thus can lead to smaller empirical margin error. This implies positive impact of depth. The combination of these two results shows that for DNN with restricted number of hidden units, increasing depth is not always good since there is a tradeoff between positive and negative impacts. These results inspire us to seek alternative ways to achieve positive impact of depth, e. g. , imposing margin-based penalty terms to cross entropy loss so as to reduce empirical margin error without increasing depth. Our experiments show that in this way, we achieve signiﬁcantly better test performance.

PDF Details