Author name cluster

Sen Wu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

1 author row

AAAI Conference 2026 Conference Paper

SALR: Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

Longteng Zhang
Sen Wu
Shuai Hou
Zhengyu Qing
Zhuo Zheng
Danning Ke
Qihong Lin
Qiang Wang

Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments. Low-rank Adaptation (LoRA) reduces trainable parameters by factorizing weight updates, yet the underlying dense weights still impose high storage and computation costs. Magnitude-based pruning can yield sparse models but typically degrades LoRA’s performance when applied naively. In this paper, we introduce SALR (Sparsity-Aware Low-Rank Representation), a novel fine-tuning paradigm that unifies low-rank adaptation with sparse pruning under a rigorous mean-squared-error framework. We prove that statically pruning only the frozen base weights minimizes the pruning error bound, and we recover the discarded residual information via a truncated-SVD low-rank adapter, which provably reduces per-entry MSE by a factor of (1 - r/min(d, k)). To maximize hardware efficiency, we fuse multiple low-rank adapters into a single concatenated GEMM, and we adopt a bitmap-based encoding with a two-stage pipelined decoding + GEMM design to achieve true model compression and speedup. Empirically, SALR attains 50% sparsity on various LLMs while matching the performance of LoRA on GSM8K and MMLU, reduces model size by 2x, and delivers up to a 1.7x inference speedup.

PDF Details DOI

JMLR Journal 2025 Journal Article

Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers

Fan Yang
Hongyang R. Zhang
Sen Wu
Christopher Re
Weijie J. Su

The problem of learning one task using samples from another task is central to transfer learning. In this paper, we focus on answering the following question: when does combining the samples from two related tasks perform better than learning with one target task alone? This question is motivated by an empirical phenomenon known as negative transfer often observed in transfer learning practice. While the transfer effect from one task to another depends on factors such as their sample sizes and the spectrum of their covariance matrices, precisely quantifying this dependence has remained a challenging problem. In order to compare a transfer learning estimator to single-task learning, one needs to compare the risks between the two estimators precisely. Further, the comparison depends on the distribution shifts between the two tasks. This paper applies recent developments of random matrix theory to tackle this challenge in a high-dimensional linear regression setting with two tasks. We provide precise high-dimensional asymptotics for the bias and variance of a classical hard parameter sharing (HPS) estimator in the proportional limit, when the sample sizes of both tasks increase proportionally with dimension at fixed ratios. The precise asymptotics apply to various types of distribution shifts, including covariate shifts, model shifts, and combinations of both. We illustrate these results in a random-effects model to mathematically prove a phase transition from positive to negative transfer as the number of source task samples increases. One insight from the analysis is that a rebalanced HPS estimator, which downsizes the source task when the model shift is high, achieves the minimax optimal rate. The finding regarding phase transition also applies to multiple tasks when feature covariates are shared across all tasks. Simulations validate the accuracy of the high-dimensional asymptotics for finite dimensions. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2025. ( edit, beta )

PDF Details

IS Journal 2020 Journal Article

Multiangle P2P Borrower Characterization Analytics by Attributes Partition Considering Business Process

Shuaiqi Liu
Sen Wu

In the research of P2P lending data, the study of borrower characteristics is of great value for the establishment of target customers and risk management. Because of high dimensionality, mixed attributes, different importance, and different generation time of information, P2P lending data often leads to the mining results unable to reflect the important borrower characteristics that affect the approval results and the approval loan amount. In this article, we are the first to propose the attributes partition of lending data considering the business process to classify variables into different types. Furthermore, we propose a multiangle data mining method for lending data by attributes partition considering the business process to discover the characteristics of P2P borrowers from multiple perspectives. Experimental results on the real dataset demonstrate that the method depicts the important characteristics of borrowers that affect the approval results and the loan amount, makes the research on P2P borrower characteristics more comprehensive and specific, and provides new ideas for the research on high-dimensional lending data.

Details DOI

NeurIPS Conference 2019 Conference Paper

Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices

Vincent Chen
Sen Wu
Alexander Ratner
Jen Weng
Christopher Ré

In real-world machine learning applications, data subsets correspond to especially critical outcomes: vulnerable cyclist detections are safety-critical in an autonomous driving task, and "question" sentences might be important to a dialogue agent's language understanding for product purposes. While machine learning models can achieve quality performance on coarse-grained metrics like F1-score and overall accuracy, they may underperform on these critical subsets---we define these as slices, the key abstraction in our approach. To address slice-level performance, practitioners often train separate "expert" models on slice subsets or use multi-task hard parameter sharing. We propose Slice-based Learning, a new programming model in which the slicing function (SF), a programmer abstraction, is used to specify additional model capacity for each slice. Any model can leverage SFs to learn slice-specific representations, which are combined with an attention mechanism to make slice-aware predictions. We show that our approach improves over baselines in terms of computational complexity and slice-specific performance by up to 19. 0 points, and overall performance by up to 4. 6 F1 points on applications spanning natural language understanding and computer vision benchmarks as well as production-scale industrial systems.

PDF Details

TIST Journal 2017 Journal Article

Multi-Hypergraph Consistent Sparse Coding

Xiaodong Feng
Sen Wu
Wenjun Zhou

Sparse representation has been a powerful technique for modeling high-dimensional data. As an unsupervised technique to extract sparse representations, sparse coding encodes the original data into a new sparse code space and simultaneously learns a dictionary representing high-level semantics. Existing methods have considered local manifold within high-dimensional data using graph/hypergraph Laplacian regularization, and more from the manifold could be utilized to improve the performance. In this article, we propose to further regulate the sparse coding so that the learned sparse codes can well reconstruct the hypergraph structure. In particular, we add a novel hypergraph consistency regularization term (HC) by minimizing the reconstruction error of the hypergraph incidence or weight matrix. Moreover, we extend the HC term to multi-hypergraph consistent sparse coding (MultiCSC) and automatically select the optimal manifold structure under the multi-hypergraph learning framework. We show that the optimization of MultiCSC can be solved efficiently, and that several existing sparse coding methods can fit into the general framework of MultiCSC as special cases. As a case study, hypergraph incidence consistent sparse coding is applied to perform semi-auto image tagging, demonstrating the effectiveness of hypergraph consistency regulation. We perform further experiments using MultiCSC for image clustering, which outperforms a number of baselines.

Details DOI

NeurIPS Conference 2016 Conference Paper

Data Programming: Creating Large Training Sets, Quickly

Alexander Ratner
Christopher De Sa
Sen Wu
Daniel Selsam
Christopher Ré

Large labeled training sets are the critical building blocks of supervised learning methods and are key enablers of deep learning techniques. For some applications, creating labeled training sets is the most time-consuming and expensive part of applying machine learning. We therefore propose a paradigm for the programmatic creation of training sets called data programming in which users provide a set of labeling functions, which are programs that heuristically label subsets of the data, but that are noisy and may conflict. By viewing these labeling functions as implicitly describing a generative model for this noise, we show that we can recover the parameters of this model to "denoise" the generated training set, and establish theoretically that we can recover the parameters of these generative models in a handful of settings. We then show how to modify a discriminative loss function to make it noise-aware, and demonstrate our method over a range of discriminative models including logistic regression and LSTMs. Experimentally, on the 2014 TAC-KBP Slot Filling challenge, we show that data programming would have led to a new winning score, and also show that applying data programming to an LSTM model leads to a TAC-KBP score almost 6 F1 points over a state-of-the-art LSTM baseline (and into second place in the competition). Additionally, in initial user studies we observed that data programming may be an easier way for non-experts to create machine learning models when training data is limited or unavailable.

PDF Details

EAAI Journal 2016 Journal Article

Social network regularized Sparse Linear Model for Top-N recommendation

Xiaodong Feng
Ankit Sharma
Jaideep Srivastava
Sen Wu
Zhiwei Tang

Social recommendation techniques have been developed to employ user׳s social connections for both rating prediction and Top-N recommendation. However, they are mostly using social network enhanced matrix factorization (MF) where the objective is to minimize the prediction error of rating scores, which makes it impractical and unsuccessful for Top-N recommendation. This paper thus focuses on developing more effective methods to utilize social network information for Top-N recommendation. Social network regularized Sparse LInear Model (SocSLIM) with its extensions incorporating local learning (LocSocSLIM) to improve efficiency are proposed. SocSLIM learns sparse coefficient matrix for users by solving a sparse representation problem over user-item rating/purchase matrix and user–user social network׳s adjacency matrix at the same time by sharing coefficient matrix. The coefficient matrix is used to predict the recommendation scores, which are then combined with a proposed item based Distance regularized Sparse LInear Model (DSLIM) to generate recommendations for the users. The experimental results demonstrate that SocSLIM effectively uses the social information to outperform the state-of-the-art methods by at least 12%. Moreover, the local weight learning extension LocSocSLIM significantly improves the efficiency up to 10 times as compared to SocSLIM as the original SLIM while achieving the close performance guarantees.

Details DOI