Author name cluster

Xiangyu He

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

ICML Conference 2025 Conference Paper

FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing

Yingying Deng
Xiangyu He
Changwang Mei
Peisong Wang
Fan Tang

Though Rectified Flows (ReFlows) with distillation offer a promising way for fast sampling, its fast inversion transforms images back to structured noise for recovery and following editing remains unsolved. This paper introduces FireFlow, an embarrassingly simple yet effective zero-shot approach that inherits the startling capacity of ReFlow-based models (such as FLUX) in generation while extending its capabilities to accurate inversion and editing in 8 steps. We first demonstrate that a carefully designed numerical solver is pivotal for ReFlow inversion, enabling accurate inversion and reconstruction with the precision of a second-order solver while maintaining the practical efficiency of a first-order Euler method. This solver achieves a $3\times$ runtime speedup compared to state-of-the-art ReFlow inversion and editing techniques while delivering smaller reconstruction errors and superior editing results in a training-free mode. The code is available at this-URL.

Details

YNIMG Journal 2025 Journal Article

The cortical architecture representing the linguistic hierarchy of the conversational speech

Ruhuiya Aili
Siyuan Zhou
Xinran Xu
Xiangyu He
Chunming Lu

Recent studies demonstrate that the brain parses natural language into smaller units represented in lower-order regions and larger units in higher-order regions. Most of these studies, however, have been conducted on unidirectional narrative speech, leaving the linguistic hierarchy and its cortical representation in bidirectional conversational speech unexplored. To address this gap, we simultaneously measured brain activity from two individuals using functional near-infrared spectroscopy (fNIRS) hyperscanning while they engaged in a naturalistic conversation. Using a Pre-trained Language Model (PLM) and Representational Similarity Analysis (RSA), we demonstrated that conversational speech, jointly produced by two interlocutors in a turn-taking manner, exhibits a linguistic hierarchy, characterized by a boundary effect between linguistic units and an incremental context effect. Furthermore, a gradient pattern of shared cortical representation of the linguistic hierarchy was identified at the dyadic rather than the individual level. Interpersonal neural synchronization (INS) in the left superior temporal cortex was associated with turn representation, whereas INS in the medial prefrontal cortex was linked to topic representation. These findings further validated the distinctiveness of linguistic units of different sizes. Together, our results provide original evidence for the linguistic hierarchy and the underlying cortical architecture during a naturalistic conversation, extending the hierarchical nature of natural language from unidirectional narrative speech to bidirectional conversational speech.

Details DOI

AAAI Conference 2023 Conference Paper

Code-Aware Cross-Program Transfer Hyperparameter Optimization

Zijia Wang
Xiangyu He
Kehan Chen
Chen Lin
Jinsong Su

Hyperparameter tuning is an essential task in automatic machine learning and big data management. To accelerate tuning, many recent studies focus on augmenting BO, the primary hyperparameter tuning strategy, by transferring information from other tuning tasks. However, existing studies ignore program similarities in their transfer mechanism, thus they are sub-optimal in cross-program transfer when tuning tasks involve different programs. This paper proposes CaTHPO, a code-aware cross-program transfer hyperparameter optimization framework, which makes three improvements. (1) It learns code-aware program representation in a self-supervised manner to give an off-the-shelf estimate of program similarities. (2) It adjusts the surrogate and AF in BO based on program similarities, thus the hyperparameter search is guided by accumulated information across similar programs. (3) It presents a safe controller to dynamically prune undesirable sample points based on tuning experiences of similar programs. Extensive experiments on tuning various recommendation models and Spark applications have demonstrated that CatHPO can steadily obtain better and more robust hyperparameter performances within fewer samples than state-of-the-art competitors.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning

Yanpeng Sun
Qiang Chen
Xiangyu He
Jian Wang
Haocheng Feng
Junyu Han
Errui Ding
Jian Cheng

Freezing the pre-trained backbone has become a standard paradigm to avoid overfitting in few-shot segmentation. In this paper, we rethink the paradigm and explore a new regime: {\em fine-tuning a small part of parameters in the backbone}. We present a solution to overcome the overfitting problem, leading to better model generalization on learning novel classes. Our method decomposes backbone parameters into three successive matrices via the Singular Value Decomposition (SVD), then {\em only fine-tunes the singular values} and keeps others frozen. The above design allows the model to adjust feature representations on novel classes while maintaining semantic clues within the pre-trained backbone. We evaluate our {\em Singular Value Fine-tuning (SVF)} approach on various few-shot segmentation methods with different backbones. We achieve state-of-the-art results on both Pascal-5$^i$ and COCO-20$^i$ across 1-shot and 5-shot settings. Hopefully, this simple baseline will encourage researchers to rethink the role of backbone fine-tuning in few-shot settings.

PDF Details

AAAI Conference 2022 Conference Paper

Towards Fully Sparse Training: Information Restoration with Spatial Similarity

Weixiang Xu
Xiangyu He
Ke Cheng
Peisong Wang
Jian Cheng

The 2: 4 structured sparsity pattern released by NVIDIA Ampere architecture, requiring four consecutive values containing at least two zeros, enables doubling math throughput for matrix multiplications. Recent works mainly focus on inference speedup via 2: 4 sparsity while training acceleration has been largely overwhelmed where backpropagation consumes around 70% of the training time. However, unlike inference, training speedup with structured pruning is nontrivial due to the need to maintain the fidelity of gradients and reduce the additional overhead of performing 2: 4 sparsity online. For the first time, this article proposes fully sparse training (FST) where ‘fully’ indicates that ALL matrix multiplications in forward/backward propagation are structurally pruned while maintaining accuracy. To this end, we begin with saliency analysis, investigating the sensitivity of different sparse objects to structured pruning. Based on the observation of spatial similarity among activations, we propose pruning activations with fixed 2: 4 masks. Moreover, an Information Restoration block is proposed to retrieve the lost information, which can be implemented by efficient gradient-shift operation. Evaluation of accuracy and efficiency shows that we can achieve 2× training acceleration with negligible accuracy degradation on challenging large-scale classification and detection tasks.

PDF Details

IJCAI Conference 2020 Conference Paper

Soft Threshold Ternary Networks

Weixiang Xu
Xiangyu He
Tianli Zhao
Qinghao Hu
Peisong Wang
Jian Cheng

Large neural networks are difficult to deploy on mobile devices because of intensive computation and storage. To alleviate it, we study ternarization, a balance between efficiency and accuracy that quantizes both weights and activations into ternary values. In previous ternarized neural networks, a hard threshold Δ is introduced to determine quantization intervals. Although the selection of Δ greatly affects the training results, previous works estimate Δ via an approximation or treat it as a hyper-parameter, which is suboptimal. In this paper, we present the Soft Threshold Ternary Networks (STTN), which enables the model to automatically determine quantization intervals instead of depending on a hard threshold. Concretely, we replace the original ternary kernel with the addition of two binary kernels at training time, where ternary values are determined by the combination of two corresponding binary values. At inference time, we add up the two binary kernels to obtain a single ternary kernel. Our method dramatically outperforms current state-of-the-arts, lowering the performance gap between full-precision networks and extreme low bit networks. Experiments on ImageNet with AlexNet (Top-1 55. 6%), ResNet-18 (Top-1 66. 2%) achieves new state-of-the-art.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Sparsity-Inducing Binarized Neural Networks

Peisong Wang
Xiangyu He
Gang Li
Tianli Zhao
Jian Cheng

Binarization of feature representation is critical for Binarized Neural Networks (BNNs). Currently, sign function is the commonly used method for feature binarization. Although it works well on small datasets, the performance on ImageNet remains unsatisﬁed. Previous methods mainly focus on minimizing quantization error, improving the training strategies and decomposing each convolution layer into several binary convolution modules. However, whether sign is the only option for binarization has been largely overlooked. In this work, we propose the Sparsity-inducing Binarized Neural Network (Si-BNN), to quantize the activations to be either 0 or +1, which introduces sparsity into binary representation. We further introduce trainable thresholds into the backward function of binarization to guide the gradient propagation. Our method dramatically outperforms current state-ofthe-arts, lowering the performance gap between full-precision networks and BNNs on mainstream architectures, achieving the new state-of-the-art on binarized AlexNet (Top-1 50. 5%), ResNet-18 (Top-1 59. 7%), and VGG-Net (Top-1 63. 2%). At inference time, Si-BNN still enjoys the high efﬁciency of exclusive-not-or (xnor) operations.

PDF Details

ICML Conference 2020 Conference Paper

Towards Accurate Post-training Network Quantization via Bit-Split and Stitching

Peisong Wang
Qiang Chen 0007
Xiangyu He
Jian Cheng 0001

Network quantization is essential for deploying deep models to IoT devices due to its high efficiency. Most existing quantization approaches rely on the full training datasets and the time-consuming fine-tuning to retain accuracy. Post-training quantization does not have these problems, however, it has mainly been shown effective for 8-bit quantization due to the simple optimization strategy. In this paper, we propose a Bit-Split and Stitching framework (Bit-split) for lower-bit post-training quantization with minimal accuracy degradation. The proposed framework is validated on a variety of computer vision tasks, including image classification, object detection, instance segmentation, with various network architectures. Specifically, Bit-split can achieve near-original model performance even when quantizing FP32 models to INT3 without fine-tuning.

Details