Author name cluster

Bei Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

1 author row

AAAI Conference 2026 Conference Paper

DesireKV: Decoupling Sensitivity and Importance for Reasoning-Aware KV Cache Compression

Pengyu Cheng
Jiacheng Wang
Tianle Chen
Bei Liu
Xiaofeng Hou
Jiacheng Liu

Large language models performing chain-of-thought (CoT) reasoning generate extensive intermediate sequences that consume substantial memory through key-value (KV) cache storage. Unlike conventional text generation, reasoning sequences exhibit unique characteristics, including repetitive logic patterns and low information density, making existing KV cache compression methods suboptimal. We propose DesireKV, a novel compression framework that first constructs a two-dimensional coordinate system based on attention-derived importance and outlier-based quantization sensitivity. It then applies a dedicated protection mechanism for tokens critical to the reasoning process itself. Our approach makes differentiated compression decisions: retaining important and sensitive tokens, quantizing important but insensitive tokens, and evicting unimportant tokens. Through comprehensive evaluation on reasoning benchmarks, we demonstrate that DesireKV achieves up to 2.93× throughput improvement while maintaining nearly 99% of original reasoning accuracy.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration

Tianteng Gu
Bei Liu
Bo Xiao
Ke Zeng
Jiacheng Liu
Yanmin Qian

Pruning is a widely used technique to compress large language models (LLMs) by removing unimportant weights, but it often suffers from significant performance degradation—especially under semi-structured sparsity constraints. Existing pruning methods primarily focus on estimating the importance of individual weights, which limits their ability to preserve critical capabilities of the model. In this work, we propose a new perspective: rather than merely selecting which weights to prune, we first redistribute parameter importance to make the model inherently more amenable to pruning. By minimizing the information entropy of normalized importance scores, our approach concentrates importance onto a smaller subset of weights, thereby enhancing pruning robustness. We instantiate this idea through DenoiseRotator, which applies learnable orthogonal transformations to the model’s weight matrices. Our method is model-agnostic and can be seamlessly integrated with existing pruning techniques such as Magnitude, SparseGPT, and Wanda. Evaluated on LLaMA3, Qwen2. 5, and Mistral models under 50% unstructured and 2: 4 semi-structured sparsity, DenoiseRotator consistently improves perplexity and zero-shot accuracy. For instance, on LLaMA3-70B pruned with SparseGPT at 2: 4 semi-structured sparsity, DenoiseRotator reduces the perplexity gap to the dense model by 58%, narrowing the degradation from 8. 1 to 3. 4 points.

PDF Details

YNIMG Journal 2025 Journal Article

Highly accelerated whole-brain T2 mapping using non-cartesian acquisition and model-based implicit neural representation reconstruction

Tianyi Xiao
Bei Liu
Huajun She
Yiping P. Du

OBJECTIVE: mapping of the whole brain. METHODS: quantification using the undersampled acquisition was validated in comparison with those using the fully sampled acquisition and multi-echo spin-echo. RESULTS: values was found between our technique and the reference method in prospective experiments, and our technique showed good inter-scan reproducibility and intra-scan repeatability. CONCLUSION: mapping can be acquired in 70 s with a 20-fold acceleration.

Details DOI

JBHI Journal 2025 Journal Article

PEARL: Cascaded Self-Supervised Cross-Fusion Learning for Parallel MRI Acceleration

Qingyong Zhu
Bei Liu
Zhuo-Xu Cui
Chentao Cao
Xiaomeng Yan
Yuanyuan Liu
Jing Cheng
Yihang Zhou

Supervised deep learning (SDL) methodology holds promise for accelerated magnetic resonance imaging (AMRI) but is hampered by the reliance on extensive training data. Some self-supervised frameworks, such as deep image prior (DIP), have emerged, eliminating the explicit training procedure but often struggling to remove noise and artifacts under significant degradation. This work introduces a novel self-supervised accelerated parallel MRI approach called PEARL, leveraging a multiple-stream joint deep decoder with two cross-fusion schemes to accurately reconstruct one or more target images from compressively sampled k-space. Each stream comprises cascaded cross-fusion sub-block networks (SBNs) that sequentially perform combined upsampling, 2D convolution, joint attention, ReLU activation and batch normalization (BN). Among them, combined upsampling and joint attention facilitate mutual learning between multiple-stream networks by integrating multi-parameter priors in both additive and multiplicative manners. Long-range unified skip connections within SBNs ensure effective information propagation between distant cross-fusion layers. Additionally, incorporating dual-normalized edge-orientation similarity regularization into the training loss enhances detail reconstruction and prevents overfitting. Experimental results consistently demonstrate that PEARL outperforms the existing state-of-the-art (SOTA) self-supervised AMRI technologies in various MRI cases. Notably, 5-fold $\sim$ 6-fold accelerated acquisition yields a 1 $\%$ $\sim$ 2 $\%$ improvement in SSIM $_{\mathsf{ROI}}$ and a 3 $\%$ $\sim$ 6 $\%$ improvement in PSNR $_{\mathsf{ROI}}$, along with a significant 15 $\%$ $\sim$ 20 $\%$ reduction in RLNE $_{\mathsf{ROI}}$.

Details DOI

NeurIPS Conference 2022 Conference Paper

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun
Hongwei Xue
Ruihua Song
Bei Liu
Huan Yang
Jianlong Fu

Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks. Previous studies of video-language pretraining mainly focus on short-form videos (i. e. , within 30 seconds) and sentences, leaving long-form video-language pre-training rarely explored. Directly learning representation from long-form videos and language may benefit many long-formvideo-language understanding tasks. However, it is challenging due to the difficulty of modeling long-range relationships and the heavy computational burden caused by more frames. In this paper, we introduce a Long-Form VIdeo-LAnguage pre-training model (LF-VILA) and train it on a large-scale long-form video and paragraph dataset constructed from an existing public dataset. To effectively capturethe rich temporal dynamics and to better align video and language in an efficient end-to-end manner, we introduce two novel designs in our LF-VILA model. We first propose a Multimodal Temporal Contrastive (MTC) loss to learn the temporal relation across different modalities by encouraging fine-grained alignment between long-form videos and paragraphs. Second, we propose a Hierarchical Temporal Window Attention (HTWA) mechanism to effectively capture long-range dependency while reducing computational cost in Transformer. We fine-tune the pre-trained LF-VILA model on seven downstream long-form video-language understanding tasks of paragraph-to-video retrieval and long-form video question-answering, and achieve new state-of-the-art performances. Specifically, our model achieves 16. 1% relative improvement on ActivityNet paragraph-to-video retrieval task and 2. 4% on How2QA task, respectively. We release our code, dataset, and pre-trained models at https: //github. com/microsoft/XPretrain.

PDF Details

NeurIPS Conference 2021 Conference Paper

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training

Hongwei Xue
Yupan Huang
Bei Liu
Houwen Peng
Jianlong Fu
Houqiang Li
Jiebo Luo

Vision-Language Pre-training (VLP) aims to learn multi-modal representations from image-text pairs and serves for downstream vision-language tasks in a fine-tuning fashion. The dominant VLP models adopt a CNN-Transformer architecture, which embeds images with a CNN, and then aligns images and text with a Transformer. Visual relationship between visual contents plays an important role in image understanding and is the basic for inter-modal alignment learning. However, CNNs have limitations in visual relation learning due to local receptive field's weakness in modeling long-range dependencies. Thus the two objectives of learning visual relation and inter-modal alignment are encapsulated in the same Transformer network. Such design might restrict the inter-modal alignment learning in the Transformer by ignoring the specialized characteristic of each objective. To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment. Specifically, we propose a metric named Inter-Modality Flow (IMF) to measure the interaction between vision and language modalities (i. e. , inter-modality). We also design a novel masking optimization mechanism named Masked Feature Regression (MFR) in Transformer to further promote the inter-modality learning. To the best of our knowledge, this is the first study to explore the benefit of Transformer for visual feature learning in VLP. We verify our method on a wide range of vision-language tasks, including Visual Question Answering (VQA), Visual Entailment and Visual Reasoning. Our approach not only outperforms the state-of-the-art VLP performance, but also shows benefits on the IMF metric.

PDF Details

NeurIPS Conference 2021 Conference Paper

Searching the Search Space of Vision Transformer

Minghao Chen
Kan Wu
Bolin Ni
Houwen Peng
Bei Liu
Jianlong Fu
Hongyang Chao
Haibin Ling

Vision Transformer has shown great visual representation power in substantial vision tasks such as recognition and detection, and thus been attracting fast-growing efforts on manually designing more effective architectures. In this paper, we propose to use neural architecture search to automate this process, by searching not only the architecture but also the search space. The central idea is to gradually evolve different search dimensions guided by their E-T Error computed using a weight-sharing supernet. Moreover, we provide design guidelines of general vision transformers with extensive analysis according to the space searching process, which could promote the understanding of vision transformer. Remarkably, the searched models, named S3 (short for Searching the Search Space), from the searched space achieve superior performance to recently proposed models, such as Swin, DeiT and ViT, when evaluated on ImageNet. The effectiveness of S3 is also illustrated on object detection, semantic segmentation and visual question answering, demonstrating its generality to downstream vision and vision-language tasks. Code and models will be available at https: //github. com/microsoft/Cream.

PDF Details

EAAI Journal 2020 Journal Article

Interval-valued fuzzy reasoning algorithms based on Schweizer–Sklar t-norms and its application

Minxia Luo
Ruirui Zhao
Bei Liu
Jingjing Liang

Based on the normalized Minkowski distance in Hausdorff metrics, we study the sensitivity of interval-valued Schweizer-Sklar t-norms and their corresponding residual implications. Moreover, we investigate the robustness of interval-valued fuzzy reasoning triple I algorithms based on Schweizer–Sklar operators and illustrate the feasibility of the algorithms by a numerical example. Finally, the interval-valued fuzzy reasoning triple I algorithms are applied to medical diagnosis.

Details DOI