Author name cluster

Hao Gu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

AAAI Conference 2026 Conference Paper

Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging

Qiyuan Zhu
Dezhi Li
Lujun Li
Xiaoyu Qin
Wei Li
Hao Gu
Hua Xu
Sirui Han

Large Reasoning Language Models (LRMs) have recently shown remarkable performance in complex reasoning tasks, but their extensive reasoning chains incur substantial computational overhead. To address this challenge, we propose Outlier-aware Reasoning Conciseness Adaptive Merge (ORCA), a novel plug-and-play model merging framework that leverages outlier activation patterns to fuse base models with reasoning models. Our ORCA introduces three key innovations: (1) adaptive alignment that reduces conflicts between disparate activation patterns during merging, (2) outlier-guided allocation that assigns merging coefficients proportional to each layer's reasoning importance as indicated by outlier concentrations, and (3) dynamic probe-based adjustment that adapts merging coefficients during inference based on input-specific activation characteristics. These strategies allow seamless integration into existing merging pipelines while creating unified models that maintain reasoning accuracy with significantly reduced response verbosity. Comprehensive evaluation across six benchmarks using Qwen and LLaMA models shows ORCA reduces average response length by 55% while improving accuracy by 2.4∼5.7% over existing methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging

Lujun Li
Qiyuan Zhu
Jiacheng Wang
Xiaoyu Qin
Wei Li
Hao Gu
Sirui Han
Yike Guo

Mixture of Experts (MoE) LLMs face significant obstacles due to their massive parameter scale, which imposes memory, storage, and deployment challenges. Although recent expert merging methods aim to achieve greater efficiency by consolidating several experts, they are fundamentally hindered by parameter conflicts arising from expert specialization. In this paper, we present Sub-MoE, a novel MoE compression framework via Subspace Expert Merging. Our key insight is to perform joint Singular Value Decomposition (SVD) on concatenated expert weights, reducing conflicting parameters by extracting shared U-matrices while enabling effective merging of the expert-specific V components. Specifically, Sub-MoE consists of two innovative stages: (1) Adaptive Expert Clustering, which groups functionally coherent experts via K-means clustering based on cosine similarity of expert outputs; and (2) Subspace Expert Merging, which first performs Experts Union Decomposition to derive the shared U-matrix across experts in the same group, then applies frequency-based merging for individual V-matrices, and completes expert reconstruction using the merged V-matrix. In this way, we align and fuse experts in a shared subspace. Additionally, the framework can be extended with intra-expert compression for further inference optimization. Extensive experiments on Mixtral, DeepSeek, and Qwen-1.5/3 MoE LLMs demonstrate that our Sub-MoE significantly outperforms existing expert pruning and merging methods. Notably, our Sub-MoE maintains 96%/86% of original performance with 25%/50% expert reduction on Mixtral-8×7B in zero-shot benchmarks.

PDF Details DOI

EAAI Journal 2025 Journal Article

A multi-point dam deformation prediction model based on spatiotemporal graph convolutional network

Taiqi Lu
Hao Gu
Chongshi Gu
Chenfei Shao
Dongyang Yuan

Dams are essential for services such as water supply and flood protection, making their structural health and safety crucial to prevent catastrophic failures. Monitoring dam displacement is a critical method for assessing its current condition and predicting future behavior. Most displacement monitoring models primarily focus on temporal features and the relationship between environmental factors and dam displacement, often overlooking the spatial relationships in the data. Even multi-point models, designed to handle multiple measurement locations, struggle to effectively account for the spatial coordination between these points. To address these challenges, this paper proposes a multi-point displacement monitoring model based on a variational auto-encoder (VAE) and a spatiotemporal graph convolutional network (STGCN). The graph structure is utilized to represent the coordinated deformation relationships among monitoring points, while also capturing temporal features and nonlinear relationships between environmental factors and dam displacement. The VAE model is first used to extract latent feature representations from historical monitoring data and monitoring point coordinates. The K-nearest neighbors (KNN) method is then applied to calculate the connection weights between monitoring points, constructing the adjacency matrix. Graph convolutional network (GCN) is utilized to extract spatial features, while gated recurrent units (GRU) capture temporal dependencies, enabling accurate multi-point displacement prediction. The model's effectiveness and accuracy are validated through comparisons with both single-point and multi-point models, while the impact of K-nearest neighbors and learnable position encoding on model performance is also evaluated. The results demonstrate that the proposed model significantly outperforms others, showing superior predictive accuracy and generalization capabilities.

Details DOI

ICML Conference 2025 Conference Paper

Delta Decompression for MoE-based LLMs Compression

Hao Gu
Wei Li 0286
Lujun Li 0001
Qiyuan Zhu
Mark G. Lee
Shengjie Sun
Wei Xue 0002
Yike Guo

Mixture-of-Experts (MoE) architectures in large language models (LLMs) achieve exceptional performance, but face prohibitive storage and memory requirements. To address these challenges, we present $D^2$-MoE, a new delta decompression compressor for reducing the parameters of MoE LLMs. Based on observations of expert diversity, we decompose their weights into a shared base weight and unique delta weights. Specifically, our method first merges each expert’s weight into the base weight using the Fisher information matrix to capture shared components. Then, we compress delta weights through Singular Value Decomposition (SVD) by exploiting their low-rank properties. Finally, we introduce a semi-dynamical structured pruning strategy for the base weights, combining static and dynamic redundancy analysis to achieve further parameter reduction while maintaining input adaptivity. In this way, our $D^2$-MoE successfully compacts MoE LLMs to high compression ratios without additional training. Extensive experiments highlight the superiority of our approach, with over 13% performance gains than other compressors on Mixtral|Phi-3. 5|DeepSeek|Qwen2 MoE LLMs at 40$\sim$60% compression rates. Codes are available in https: //github. com/lliai/D2MoE.

Details

EAAI Journal 2025 Journal Article

Interpretable interval prediction of dam displacement based on variational autoencoder and improved temporal fusion transformer considering solar radiation effects

Taiqi Lu
Hao Gu
Chongshi Gu
Chenfei Shao
Yiming Wang
Dongyang Yuan

Ensuring the safety of dams is critical to maintaining national economic development and social stability, requiring the implementation of accurate displacement prediction methods for early detection of structural anomalies and effective risk mitigation. However, existing statistical models primarily focus on point predictions, failing to quantify the uncertainty in displacement variations, and often neglect the critical environmental factor of solar radiation. To address these limitations, this study proposes a novel interpretable interval prediction framework that integrates solar radiation factors into an advanced hydrostatic-temperature-time (AHTT) model. A variational autoencoder (VAE) is employed to extract robust latent features from a large volume of measured temperature data, effectively reducing temperature-related noise. Subsequently, an improved temporal fusion transformer method is introduced to probabilistic dam displacement prediction. This method uses an enhanced quantile loss function based on the Huber loss to generate both point and interval predictions that dynamically reflect the prediction uncertainty. In addition, an interpretable multi-head attention module is incorporated to quantify the contribution of each environmental factor. Hyperparameter tuning of the improved temporal fusion transformer is further optimized using Bayesian optimization based on the tree-structured Parzen estimator (TPE), which improves prediction accuracy. Engineering case studies validate that the proposed model not only achieves the highest point prediction accuracy, but also provides narrower prediction intervals with the best coverage width criterion. Ablation experiments and interpretability analyses further confirm the significant impact of solar radiation on dam displacement, providing valuable insights for the development of dam displacement prediction models and risk-informed decision making.

Details DOI

ICML Conference 2025 Conference Paper

MoE-SVD: Structured Mixture-of-Experts LLMs Compression via Singular Value Decomposition

Wei Li 0286
Lujun Li 0001
Hao Gu
You-Liang Huang
Mark G. Lee
Shengjie Sun
Wei Xue 0002
Yike Guo

Mixture of Experts (MoE) architecture improves Large Language Models (LLMs) with better scaling, but its higher parameter counts and memory demands create challenges for deployment. In this paper, we present MoE-SVD, a new decomposition-based compression framework tailored for MoE LLMs without any extra training. By harnessing the power of Singular Value Decomposition (SVD), MoE-SVD addresses the critical issues of decomposition collapse and matrix redundancy in MoE architectures. Specifically, we first decompose experts into compact low-rank matrices, resulting in accelerated inference and memory optimization. In particular, we propose selective decomposition strategy by measuring sensitivity metrics based on weight singular values and activation statistics to automatically identify decomposable expert layers. Then, we share a single V-matrix across all experts and employ a top-k selection for U-matrices. This low-rank matrix sharing and trimming scheme allows for significant parameter reduction while preserving diversity among experts. Comprehensive experiments on Mixtral, Phi-3. 5, DeepSeek, and Qwen2 MoE LLMs show MoE-SVD outperforms other compression methods, achieving a 60% compression ratio and 1. 5$\times$ faster inference with minimal performance loss.

Details

ICML Conference 2025 Conference Paper

OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition

Zheng Lian 0004
Haiyang Sun 0004
Licai Sun
Haoyu Chen 0001
Lan Chen 0005
Hao Gu
Zhuofan Wen 0001
Shun Chen

Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predominantly rely on predefined emotion taxonomies, which fail to capture the inherent complexity, subtlety, and multi-appraisal nature of human emotional experiences, as demonstrated by studies in psychology and cognitive science. To overcome this limitation, we advocate for introducing the concept of open vocabulary into MER. This paradigm shift aims to enable models to predict emotions beyond a fixed label space, accommodating a flexible set of categories to better reflect the nuanced spectrum of human emotions. To achieve this, we propose a novel paradigm: Open-Vocabulary MER (OV-MER), which enables emotion prediction without being confined to predefined spaces. However, constructing a dataset that encompasses the full range of emotions for OV-MER is practically infeasible; hence, we present a comprehensive solution including a newly curated database, novel evaluation metrics, and a preliminary benchmark. By advancing MER from basic emotions to more nuanced and diverse emotional states, we hope this work can inspire the next generation of MER, enhancing its generalizability and applicability in real-world scenarios. Code and dataset are available at: https: //github. com/zeroQiaoba/AffectGPT.

Details

AAAI Conference 2025 Conference Paper

Region-Based Optimization in Continual Learning for Audio Deepfake Detection

Yujie Chen
Jiangyan Yi
Cunhang Fan
Jianhua Tao
Yong Ren
Siding Zeng
Chu Yuan Zhang
Xinrui Yan

Rapid advancements in speech synthesis and voice conversion bring convenience but also new security risks, creating an urgent need for effective audio deepfake detection. Although current models perform well, their effectiveness diminishes when confronted with the diverse and evolving nature of real-world deepfakes. To address this issue, we propose a continual learning method named Region-Based Optimization (RegO) for audio deepfake detection. Specifically, we use the Fisher information matrix to measure important neuron regions for real and fake audio detection, dividing them into four regions. First, we directly fine-tune the less important regions to quickly adapt to new tasks. Next, we apply gradient optimization in parallel for regions important only to real audio detection, and in orthogonal directions for regions important only to fake audio detection. For regions that are important to both, we use sample proportion-based adaptive gradient optimization. This region-adaptive optimization ensures an appropriate trade-off between memory stability and learning plasticity. Additionally, to address the increase of redundant neurons from old tasks, we further introduce the Ebbinghaus forgetting mechanism to release them, thereby promoting the model’s ability to learn more generalized discriminative features. Experimental results show our method achieves a 21.3 percent improvement in EER over the state-of-the-art continual learning approach RWM for audio deepfake detection. Moreover, the effectiveness of RegO extends beyond the audio deepfake detection domain, showing potential significance in other tasks, such as image recognition.

PDF Details DOI

TCS Journal 2015 Journal Article

Set covering in fuel-considered vehicle routing problems

Liang Song
Haibin Chen
Hao Gu
Hejiao Huang
Hongwei Du

The paper studies set covering in fuel-considered vehicle routing problems (FVRP). Firstly, we study the FVRP with distance constraint and time windows (FVRP-TW) whose objective is to find a set covering with the minimum cardinality, which means the number of used vehicles is minimized and hence the fuel consumption is minimized in the real logistics. We give a bicriteria approximation algorithm for this problem. Secondly, we study the set covering in the FVRP with distance constraint and constant time windows (FVRP-CTW), which has a constant number of the time windows provided by logistics companies. We give a bicriteria approximation algorithm with ( 2 + ϵ, O ( log ⁡ 1 / ϵ ) ) for this problem, in which the first term is the approximation ratio on the distance constraint and the second term is the approximation ratio on the cardinality of the covering set. Thirdly, we study the set covering in general FVRP and propose a lower bound for this problem which is based on total unimodularity. Finally, we design an algorithm framework based on the lower bound for solving the set covering in general FVRP. Simulation results demonstrate the effectiveness of the algorithms for solving the set covering in FVRP-TW and general FVRP.

Details DOI