Author name cluster

Feng Ling

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

IS Journal 2026 Journal Article

Transforming Future Data Center Operations and Management via Physical AI

Zhiwei Cao
Minghao Li
Feng Ling
Jimin Jia
Yonggang Wen
Jianxiong Yin
Simon See

Data centers (DCs) are critical for artificial intelligence (AI) and the digital economy, with AIDCs introducing new operational challenges. This research proposes a novel Physical AI (PhyAI) framework to advance DC operations and management. The system features three core modules: an industry-grade in-house DC simulation engine for high-fidelity AIDC modeling, an AI engine built on NVIDIA PhysicsNeMo for training and evaluating physics-informed machine learning (PIML) models, and a digital twin platform based on NVIDIA Omniverse. This framework enables the creation of real-time digital twins to digitalize, optimize, and automate future DC operations. A case study demonstrated its effectiveness in predicting thermal and airflow profiles for a large-scale DC in real-time, achieving a median absolute temperature prediction error of 0. 18 °C, outperforming traditional CFD/HT simulations. This emerging approach would open doors to several potential research directions for advancing PhyAI in future DC operations.

Details DOI

NeurIPS Conference 2025 Conference Paper

PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models

Tianchen Zhao
Ke Hong
Xinhao Yang
Xuefeng Xiao
Huixia Li
Feng Ling
Ruiqi Xie
Siqi Chen

In visual generation, the quadratic complexity of attention mechanisms results in high memory and computational costs, especially for longer token sequences required in high-resolution image or multi-frame video generation. To address this, prior research has explored techniques such as sparsification and quantization. However, these techniques face significant challenges under low density and reduced bitwidths. Through systematic analysis, we identify that the core difficulty stems from the dispersed and irregular characteristics of visual attention patterns. Therefore, instead of introducing specialized sparsification and quantization design to accommodate such patterns, we propose an alternative strategy: "reorganizing" the attention pattern to alleviate the challenges. Inspired by the local aggregatin nature of visual feature extraction, we design a novel P attern- A ware token R e O rdering ( PARO ) technique, which unifies the diverse attention patterns into a hardware-friendly block-wise pattern. This unification substantially simplifies and enhances both sparsification and quantization. We evaluate the performance-efficiency trade-offs of various design choices and finalize a methodology tailored for the unified pattern. Our approach, PAROAttention, achieves video and image generation with lossless metrics, and nearly identical results from full-precision (FP) baselines, while operating at notably lower density ( 20%-30% ) and bitwidth ( INT8/INT4 ), achieving a 1. 9 - 2. 7x end-to-end latency speedup.

PDF Details

ICLR Conference 2024 Conference Paper

AffineQuant: Affine Transformation Quantization for Large Language Models

Yuexiao Ma
Huixia Li
Xiawu Zheng
Feng Ling
Xuefeng Xiao 0001
Rui Wang 0089
Shilei Wen
Fei Chao 0001

The significant resource requirements associated with Large-scale Language Models (LLMs) have generated considerable interest in the development of techniques aimed at compressing and accelerating neural networks. Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for LLMs limit the optimization scope to scaling transformations between pre- and post-quantization weights. This constraint results in significant errors after quantization, particularly in low-bit configurations. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant). This approach extends the optimization scope and thus significantly minimizing quantization errors. Additionally, by employing the corresponding inverse matrix, we can ensure equivalence between the pre- and post-quantization outputs of PTQ, thereby maintaining its efficiency and generalization capabilities. To ensure the invertibility of the transformation during optimization, we further introduce a gradual mask optimization method. This method initially focuses on optimizing the diagonal elements and gradually extends to the other elements. Such an approach aligns with the Levy-Desplanques theorem, theoretically ensuring invertibility of the transformation. As a result, significant performance improvements are evident across different LLMs on diverse datasets. Notably, these improvements are most pronounced when using very low-bit quantization, enabling the deployment of large models on edge devices. To illustrate, we attain a C4 perplexity of $15.76$ (2.26$\downarrow$ vs $18.02$ in OmniQuant) on the LLaMA2-$7$B model of W$4$A$4$ quantization without overhead. On zero-shot tasks, AffineQuant achieves an average of $58.61\%$ accuracy ( $1.98\%\uparrow$ vs $56.63$ in OmniQuant) when using $4$/$4$-bit quantization for LLaMA-$30$B, which setting a new state-of-the-art benchmark for PTQ in LLMs. Codes are available at: https://github.com/bytedance/AffineQuant.

Details

ICML Conference 2024 Conference Paper

Outlier-aware Slicing for Post-Training Quantization in Vision Transformer

Yuexiao Ma
Huixia Li
Xiawu Zheng
Feng Ling
Xuefeng Xiao 0001
Rui Wang 0089
Shilei Wen
Fei Chao 0001

Post-Training Quantization (PTQ) is a vital technique for network compression and acceleration, gaining prominence as model sizes increase. This paper addresses a critical challenge in PTQ: the severe impact of outliers on the accuracy of quantized transformer architectures. Specifically, we introduce the concept of ‘reconstruction granularity’ as a novel solution to this issue, which has been overlooked in previous works. Our work provides theoretical insights into the role of reconstruction granularity in mitigating the outlier problem in transformer models. This theoretical framework is supported by empirical analysis, demonstrating that varying reconstruction granularities significantly influence quantization performance. Our findings indicate that different architectural designs necessitate distinct optimal reconstruction granularities. For instance, the multi-stage Swin Transformer architecture benefits from finer granularity, a deviation from the trends observed in ViT and DeiT models. We further develop an algorithm for determining the optimal reconstruction granularity for various ViT models, achieving state-of-the-art (SOTA) performance in PTQ. For example, applying our method to $4$-bit quantization, the Swin-Base model achieves a Top-1 accuracy of $82. 24%$ on the ImageNet classification task. This result surpasses the RepQ-ViT by $3. 92%$ ($82. 24%$ VS $78. 32%$). Similarly, our approach elevates the ViT-Small to a Top-1 accuracy of $80. 50%$, outperforming NoisyQuant by $3. 64%$ ($80. 50%$ VS $76. 86%$). Codes are available in Supplementary Materials.

Details

EAAI Journal 2021 Journal Article

Matching images and texts with multi-head attention network for cross-media hashing retrieval

Zhixin Li
Xiumin Xie
Feng Ling
Huifang Ma
Zhiping Shi

The cross-media hashing retrieval generally encodes multimedia data into a common binary hash space, which can effectively measure the correlation between samples from different modalities. However, in the cross-media retrieval, supervised methods require a lot of manual labels, which leads to the problem of high labor in practical application. Simultaneously, most unsupervised methods do not achieve good results by preserving the correlation between or within modalities. To attack these problems and further improve retrieval performance, this paper proposes an unsupervised cross-media hashing retrieval method based on multi-head attention network, which contains rich semantic information to match images and texts better. Specifically, we make use of a multi-head attention network for generating binary hash code better. At the same time, an auxiliary similarity matrix is constructed to integrate the original neighborhood information from different modalities. Therefore, this method can capture the potential relationships between inter-modal and intra-modal correlations. Furthermore, the method is unsupervised and requires no additional semantic labels, so it has the potential to achieve large-scale cross-media retrieval. In addition, two strategies of batch normalization and replacing hash code generation functions are adopted to optimize the model, and two loss functions are designed to make the performance of our method exceed that of many supervised cross-media hashing retrieval methods. Experiments on three baseline datasets show that our method performs much better than many state-of-the-art methods. The results demonstrate the effectiveness and superiority of our method.

Details DOI

YNIMG Journal 2012 Journal Article

Rearranging the world: Neural network supporting the processing of temporal connectives

Zheng Ye
Marta Kutas
Marie St. George
Martin I. Sereno
Feng Ling
Thomas F. Münte

Temporal connectives (before/after) give us the freedom to describe a sequence of events in different orders. Studies have suggested that ‘before-initiating’ sentences, in which events are expressed in an order inconsistent with their actual order of occurrence, might need additional computation(s) during comprehension. The results of independent component analysis suggest that these computations are supported by a neural network connecting the bilateral caudate nucleus with the right middle frontal gyrus, left precentral gyrus, bilateral parietal lobule and inferior temporal gyrus. Among those regions, the caudate nucleus and the left middle frontal gyrus showed greater activations for ‘before’ than ‘after’ sentences. The functional network observed in this study may support sequence learning and processing in a general sense.

Details DOI