Author name cluster

Yuhui Xu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

ICML Conference 2025 Conference Paper

Reward-Guided Speculative Decoding for Efficient LLM Reasoning

Baohao Liao
Yuhui Xu
Hanze Dong
Junnan Li 0001
Christof Monz
Silvio Savarese
Doyen Sahoo
Caiming Xiong

We introduce Reward-Guided Speculative Decoding (RSD), a novel framework aimed at improving the efficiency of inference in large language models (LLMs). RSD synergistically combines a lightweight draft model with a more powerful target model, incorporating a controlled bias to prioritize high-reward outputs, in contrast to existing speculative decoding methods that enforce strict unbiasedness. RSD employs a process reward model to evaluate intermediate decoding steps and dynamically decide whether to invoke the target model, optimizing the trade-off between computational cost and output quality. We theoretically demonstrate that a threshold-based mixture strategy achieves an optimal balance between resource utilization and performance. Extensive evaluations on challenging reasoning benchmarks, including Olympiad-level tasks, show that RSD delivers significant efficiency gains against decoding with the target model only (up to 4. 4X fewer FLOPs), while achieving significant better accuracy than parallel decoding method on average (up to +3. 5). These results highlight RSD as a robust and cost-effective approach for deploying LLMs in resource-intensive scenarios.

Details

NeurIPS Conference 2025 Conference Paper

Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis

Tianbao Xie
Jiaqi Deng
Xiaochuan Li
Junlin Yang
Haoyuan Wu
Jixuan Chen
Wenjing Hu
Xinyuan Wang

Graphical user interface (GUI) grounding, the ability to map natural language instructions to specific actions on graphical user interfaces, remains a critical bottleneck in computer use agent development. Current benchmarks oversimplify grounding tasks as short referring expressions, failing to capture the complexity of real-world interactions that require software commonsense, layout understanding, and fine-grained manipulation capabilities. To address these limitations, we introduce OSWorld-G, a comprehensive benchmark comprising 564 finely annotated samples across diverse task types including text matching, element recognition, layout understanding, and precise manipulation. Additionally, we synthesize and release the largest computer use grounding dataset Jedi, which contains 4 million examples through multi-perspective decoupling of tasks. Our multi-scale models trained on Jedi demonstrate its effectiveness by outperforming existing approaches on ScreenSpot-v2, ScreenSpot-Pro, and our OSWorld-G. Furthermore, we demonstrate that improved grounding with Jedi directly enhances agentic capabilities of general foundation models on complex computer tasks with state-of-the-art performance, improving from 23% to 51% on OSWorld. Through detailed ablation studies, we identify key factors contributing to grounding performance and verify that combining specialized data for different interface elements enables compositional generalization to novel interfaces. All benchmark, data, checkpoints, and code are open-sourced and available at https: //osworld-grounding. github. io.

PDF Details

ICLR Conference 2025 Conference Paper

ThinK: Thinner Key Cache by Query-Driven Pruning

Yuhui Xu
Zhanming Jie
Hanze Dong
Lei Wang 0185
Xudong Lu
Aojun Zhou
Amrita Saha
Caiming Xiong

Large Language Models (LLMs) have revolutionized the field of natural language processing, achieving unprecedented performance across a variety of applications. However, their increased computational and memory demands present significant challenges, especially when handling long sequences. This paper focuses on the long-context scenario, addressing the inefficiencies in KV cache memory consumption during inference. Unlike existing approaches that optimize the memory based on the sequence length, we identify substantial redundancy in the channel dimension of the KV cache, as indicated by an uneven magnitude distribution and a low-rank structure in the attention weights. In response, we propose ThinK, a novel query-dependent KV cache pruning method designed to minimize attention weight loss while selectively pruning the least significant channels. Our approach not only maintains or enhances model accuracy but also achieves a reduction in KV cache memory costs by over 20\% compared with vanilla KV cache eviction and quantization methods. For instance, ThinK integrated with KIVI can achieve $2.8\times$ peak memory reduction while maintaining nearly the same quality, enabling a batch size increase from 4$\times$ (with KIVI alone) to 5$\times$ when using a single GPU. Extensive evaluations on the LLaMA and Mistral models across various long-sequence datasets verified the efficiency of ThinK. Our code has been made available at https://github.com/SalesforceAIResearch/ThinK.

Details

EAAI Journal 2024 Journal Article

Dynamic time scales ensemble framework for similarity-based remaining useful life prediction under multiple failure modes

Yuhui Xu
Tangbin Xia
Dong Wang
Zhen Chen
Ershun Pan
Lifeng Xi

In modern industry, the stochastic degradation of mechanical equipment typically involves multiple failure modes, which heavily affects the reliability of the remaining useful life (RUL) prediction. The similarity-based methods have been widely deployed in RUL prediction due to their flexibility, but it is still challenging to accurately identify similar degradation trajectories under varying failure modes. The obstacles lie in the interference of reference trajectories under different degradation states and the insufficiency of measuring trajectory trends. Therefore, this paper proposes a dynamic scales ensemble method based on the mean removal Canberra distance with failure identification (FI-MRC-DSE) for similarity-based prognosis. Firstly, a gated recurrent unit autoencoder network is employed to adaptively extract failure features from multi-dimensional monitoring data to support the targeted selection of reference trajectories. Then, the similarity matching is performed based on the proposed MRC distance instead of the commonly used Euclidean distance, enhancing the perception of degradation trends. Finally, the matching results across multiple time scales, which are dynamically determined by the instance's degradation state, are integrated to obtain the predicted RUL. It effectively overcomes the insufficient utilization of trajectory caused by the single time scale. In the experiments, the superiority of our developed similarity-based FI-MRC-DSE method is demonstrated by comparison with the state-of-the-art similarity-based methods. The effectiveness analyses and the ablation study show that all three key components contribute to accurate prognosis under multiple failure modes.

Details DOI

ICML Conference 2024 Conference Paper

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models

Xudong Lu
Aojun Zhou
Yuhui Xu
Renrui Zhang
Peng Gao 0007
Hongsheng Li 0001

Large Language Models (LLMs) have become pivotal in advancing the field of artificial intelligence, yet their immense sizes pose significant challenges for both fine-tuning and deployment. Current post-training pruning methods, while reducing the sizes of LLMs, often fail to maintain their original performance. To address these challenges, this paper introduces SPP, a S parsity- P reserved P arameter-efficient fine-tuning method. Different from existing post-training pruning approaches that struggle with performance retention, SPP proposes to employ lightweight learnable column and row matrices to optimize sparse LLM weights, keeping the structure and sparsity of pruned pre-trained models intact. By element-wise multiplication and residual addition, SPP ensures the consistency of model sparsity pattern and ratio during both training and weight-merging processes. We demonstrate the effectiveness of SPP by applying it to the LLaMA and LLaMA-2 model families with recent post-training pruning methods. Our results show that SPP significantly enhances the performance of models with different sparsity patterns (i. e. unstructured and N: M sparsity), especially for those with high sparsity ratios (e. g. 75%), making it a promising solution for the efficient fine-tuning of sparse LLMs. Code will be made available at https: //github. com/Lucky-Lance/SPP.

Details

AAAI Conference 2021 Conference Paper

Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks

Xin Chen
Lingxi Xie
Jun Wu
Longhui Wei
Yuhui Xu
Qi Tian

Neural architecture search has attracted wide attentions in both academia and industry. To accelerate it, researchers proposed weight-sharing methods which first train a super-network to reuse computation among different operators, from which exponentially many sub-networks can be sampled and efficiently evaluated. These methods enjoy great advantages in terms of computational costs, but the sampled sub-networks are not guaranteed to be estimated precisely unless an individual training process is taken. This paper owes such inaccuracy to the inevitable mismatch between assembled network layers, so that there is a random error term added to each estimation. We alleviate this issue by training a graph convolutional network to fit the performance of sampled sub-networks so that the impact of random errors becomes minimal. With this strategy, we achieve a higher rank correlation coefficient in the selected set of candidates, which consequently leads to better performance of the final architecture. In addition, our approach also enjoys the flexibility of being used under different hardware constraints, since the graph convolutional network has provided an efficient lookup table of the performance of architectures in the entire search space.

PDF Details

IJCAI Conference 2020 Conference Paper

TRP: Trained Rank Pruning for Efficient Deep Neural Networks

Yuhui Xu
Yuxi Li
Shuai Zhang
Wei Wen
Botao Wang
Yingyong Qi
Yiran Chen
Weiyao Lin

To enable DNNs on edge devices like mobile phones, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several previous works attempted to directly approximate a pre-trained model by low-rank decomposition; however, small approximation errors in parameters can ripple over a large prediction loss. As a result, performance usually drops significantly and a sophisticated effort on fine-tuning is required to recover accuracy. Apparently, it is not optimal to separate low-rank approximation from training. Unlike previous works, this paper integrates low rank approximation and regularization into the training process. We propose Trained Rank Pruning (TRP), which alternates between low rank approximation and training. TRP maintains the capacity of the original network while imposing low-rank constraints during training. A nuclear regularization optimized by stochastic sub-gradient descent is utilized to further promote low rank in TRP. The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss, thus eliminating the fine-tuning process after low rank decomposition. The proposed method is comprehensively evaluated on CIFAR-10 and ImageNet, outperforming previous compression methods using low rank approximation.

PDF Details DOI

AAAI Conference 2018 Conference Paper

Deep Neural Network Compression With Single and Multiple Level Quantization

Yuhui Xu
Yongzhuang Wang
Aojun Zhou
Weiyao Lin
Hongkai Xiong

Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufﬁciently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network quantization (SLQ) for high-bit quantization and multi-level network quantization (MLQ) for extremely low-bit quantization (ternary). We are the ﬁrst to consider the network quantization from both width and depth level. In the width level, parameters are divided into two parts: one for quantization and the other for re-training to eliminate the quantization loss. SLQ leverages the distribution of the parameters to improve the width level. In the depth level, we introduce incremental layer compensation to quantize layers iteratively which decreases the quantization loss in each iteration. The proposed approaches are validated with extensive experiments based on the state-of-the-art neural networks including AlexNet, VGG-16, GoogleNet and ResNet-18. Both SLQ and MLQ achieve impressive results.

PDF Details