Author name cluster

Li Shang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

EAAI Journal 2025 Journal Article

Deep learning-based modelling of polyvinyl chloride tube-confined concrete columns under different load eccentricities

Li Shang
Haytham F. Isleem
Mostafa M. Alsaadawi

This study presents a deep learning-based framework for predicting the load-carrying capacity of polyvinyl chloride (PVC) tube-confined concrete columns under various loading conditions. A comprehensive dataset of 200 samples was generated using finite element modeling, incorporating key parameters such as PVC tube thickness, concrete strength, and load eccentricity. Several machine learning algorithms, including Linear Regression (LR), Support Vector Regression (SVR), Decision Tree (DT), Random Forest (RF), Gradient Boosting (GB), and a novel hybrid Transformer-Convolutional Neural Network (Transformer-CNN) model, were employed for the prediction task. The results demonstrate that the proposed Transformer-CNN model outperforms traditional methods, achieving the lowest root mean squared error of 27. 15 kN and the highest coefficient of determination value of 0. 9875. The model's robustness was further validated using cross-validation techniques, ensuring its reliability for practical applications. To facilitate usability, a Python-based graphical user interface (GUI) was developed, enabling engineers to apply the model efficiently in real-world scenarios. This study highlights the potential of deep learning in advancing the design and analysis of PVC-confined concrete columns, offering a more accurate and efficient alternative to conventional methods.

Details DOI

NeurIPS Conference 2024 Conference Paper

Once Read is Enough: Domain-specific Pretraining-free Language Models with Cluster-guided Sparse Experts for Long-tail Domain Knowledge

Fang Dong
Mengyi Chen
Jixian Zhou
Yubin Shi
Yixuan Chen
Mingzhi Dong
Yujiang Wang
Dongsheng Li

Language models (LMs) only pretrained on a general and massive corpus usually cannot attain satisfying performance on domain-specific downstream tasks, and hence, applying domain-specific pretraining to LMs is a common and indispensable practice. However, domain-specific pretraining can be costly and time-consuming, hindering LMs' deployment in real-world applications. In this work, we consider the incapability to memorize domain-specific knowledge embedded in the general corpus with rare occurrences and long-tail distributions as the leading cause for pretrained LMs' inferior downstream performance. Analysis of Neural Tangent Kernels (NTKs) reveals that those long-tail data are commonly overlooked in the model's gradient updates and, consequently, are not effectively memorized, leading to poor domain-specific downstream performance. Based on the intuition that data with similar semantic meaning are closer in the embedding space, we devise a Cluster-guided Sparse Expert (CSE) layer to actively learn long-tail domain knowledge typically neglected in previous pretrained LMs. During pretraining, a CSE layer efficiently clusters domain knowledge together and assigns long-tail knowledge to designate extra experts. CSE is also a lightweight structure that only needs to be incorporated in several deep layers. With our training strategy, we found that during pretraining, data of long-tail knowledge gradually formulate isolated, outlier clusters in an LM's representation spaces, especially in deeper layers. Our experimental results show that only pretraining CSE-based LMs is enough to achieve superior performance than regularly pretrained-finetuned LMs on various downstream tasks, implying the prospects of domain-specific-pretraining-free language models.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models

Yubin Shi
Yixuan Chen
Mingzhi Dong
Xiaochen Yang
Dongsheng Li
Yujiang Wang
Robert Dick
Qin Lv

Despite their prevalence in deep-learning communities, over-parameterized models convey high demands of computational costs for proper training. This work studies the fine-grained, modular-level learning dynamics of over-parameterized models to attain a more efficient and fruitful training strategy. Empirical evidence reveals that when scaling down into network modules, such as heads in self-attention models, we can observe varying learning patterns implicitly associated with each module's trainability. To describe such modular-level learning capabilities, we introduce a novel concept dubbed modular neural tangent kernel (mNTK), and we demonstrate that the quality of a module's learning is tightly associated with its mNTK's principal eigenvalue $\lambda_{\max}$. A large $\lambda_{\max}$ indicates that the module learns features with better convergence, while those miniature ones may impact generalization negatively. Inspired by the discovery, we propose a novel training strategy termed Modular Adaptive Training (MAT) to update those modules with their $\lambda_{\max}$ exceeding a dynamic threshold selectively, concentrating the model on learning common features and ignoring those inconsistent ones. Unlike most existing training schemes with a complete BP cycle across all network modules, MAT can significantly save computations by its partially-updating strategy and can further improve performance. Experiments show that MAT nearly halves the computational cost of model training and outperforms the accuracy of baselines.

PDF Details

AAAI Conference 2017 Conference Paper

ERMMA: Expected Risk Minimization for Matrix Approximation-based Recommender Systems

Dongsheng Li
Chao Chen
Qin Lv
Li Shang
Stephen Chu
Hongyuan Zha

Matrix approximation (MA) is one of the most popular techniques in today’s recommender systems. In most MA-based recommender systems, the problem of risk minimization should be deﬁned, and how to achieve minimum expected risk in model learning is one of the most critical problems to recommendation accuracy. This paper addresses the expected risk minimization problem, in which expected risk can be bounded by the sum of optimization error and generalization error. Based on the uniform stability theory, we propose an expected risk minimized matrix approximation method (ER- MMA), which is designed to achieve better tradeoff between optimization error and generalization error in order to reduce the expected risk of the learned MA models. Theoretical analysis shows that ERMMA can achieve lower expected risk bound than existing MA methods. Experimental results on the MovieLens and Netﬂix datasets demonstrate that ERMMA outperforms six state-of-the-art MA-based recommendation methods in both rating prediction problem and item ranking problem.

PDF Details

AAAI Conference 2017 Conference Paper

GLOMA: Embedding Global Information in Local Matrix Approximation Models for Collaborative Filtering

Chao Chen
Dongsheng Li
Qin Lv
Junchi Yan
Li Shang
Stephen Chu

Recommender systems have achieved great success in recent years, and matrix approximation (MA) is one of the most popular techniques for collaborative ﬁltering (CF) based recommendation. However, a major issue is that MA methods perform poorly at detecting strong localized associations among closely related users and items. Recently, some MA-based CF methods adopt clustering methods to discover meaningful user-item subgroups and perform ensemble on different clusterings to improve the recommendation accuracy. However, ensemble learning suffers from lower efﬁciency due to the increased overall computation overhead. In this paper, we propose GLOMA, a new clustering-based matrix approximation method, which can embed global information in local matrix approximation models to improve recommendation accuracy. In GLOMA, a MA model is ﬁrst trained on the entire data to capture global information. The global MA model is then utilized to guide the training of cluster-based local MA models, such that the local models can detect strong localized associations shared within clusters and at the same time preserve global associations shared among all users/items. Evaluation results using MovieLens and Netﬂix datasets demonstrate that, by integrating global information in local models, GLOMA can outperform ﬁve state-of-the-art MA-based CF methods in recommendation accuracy while achieving descent efﬁciency.

PDF Details

AAAI Conference 2013 Conference Paper

A Maximum K-Min Approach for Classification

Mingzhi Dong
Liang Yin
Weihong Deng
Li Shang
Jun Guo
Honggang Zhang

In this paper, a general Maximum K-Min approach for classiﬁcation is proposed. With the physical meaning of optimizing the classiﬁcation conﬁdence of the K worst instances, Maximum K-Min Gain/Minimum K- Max Loss (MKM) criterion is introduced. To make the original optimization problem with combinational number of constraints computationally tractable, the optimization techniques are adopted and a general compact representation lemma for MKM Criterion is summarized. Based on the lemma, a Nonlinear Maximum K- Min (NMKM) classiﬁer and a Semi-supervised Maximum K-Min (SMKM) classiﬁer are presented for traditional classiﬁcation task and semi-supervised classi- ﬁcation task respectively. Based on the experiment results of publicly available datasets, our Maximum K- Min methods have achieved competitive performance when comparing against Hinge Loss classiﬁers.

PDF Details