Author name cluster

Xing Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

ICLR Conference 2025 Conference Paper

Circuit Transformer: A Transformer That Preserves Logical Equivalence

Xihan Li 0001
Xing Li 0023
Lei Chen 0002
Xing Zhang
Mingxuan Yuan
Jun Wang 0012

Implementing Boolean functions with circuits consisting of logic gates is fundamental in digital computer design. However, the implemented circuit must be exactly equivalent, which hinders generative neural approaches on this task due to their occasionally wrong predictions. In this study, we introduce a generative neural model, the “Circuit Transformer”, which eliminates such wrong predictions and produces logic circuits strictly equivalent to given Boolean functions. The main idea is a carefully designed decoding mechanism that builds a circuit step-by-step by generating tokens, which has beneficial “cutoff properties” that block a candidate token once it invalidate equivalence. In such a way, the proposed model works similar to typical LLMs while logical equivalence is strictly preserved. A Markov decision process formulation is also proposed for optimizing certain objectives of circuits. Experimentally, we trained an 88-million-parameter Circuit Transformer to generate equivalent yet more compact forms of input circuits, outperforming existing neural approaches on both synthetic and real world benchmarks, without any violation of equivalence constraints. Code: https://github.com/snowkylin/circuit-transformer

Details

NeurIPS Conference 2024 Conference Paper

Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation

Wei Dong
Yuan Sun
Yiting Yang
Xing Zhang
Zhijun Lin
Qingsen Yan
Haokui Zhang
Peng Wang

A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adaptation matrix. This matrix is decomposed into a product of down-projection and up-projection matrices, with the bottleneck dimensionality being crucial for reducing the number of learnable parameters, as exemplified by prevalent methods like LoRA and Adapter. However, these low-rank strategies typically employ a fixed bottleneck dimensionality, which limits their flexibility in handling layer-wise variations. To address this limitation, we propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix. We utilize Householder transformations to construct orthogonal matrices that efficiently mimic the unitary matrices, requiring only a vector. The diagonal values are learned in a layer-wise manner, allowing them to flexibly capture the unique properties of each layer. This approach enables the generation of adaptation matrices with varying ranks across different layers, providing greater flexibility in adapting pre-trained models. Experiments on standard downstream vision tasks demonstrate that our method achieves promising fine-tuning performance.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Toward Robust Diagnosis: A Contour Attention Preserving Adversarial Defense for COVID-19 Detection

Kun Xiang
Xing Zhang
Jinwen She
Jinpeng Liu
Haohan Wang
Shiqi Deng
Shancheng Jiang

As the COVID-19 pandemic puts pressure on healthcare systems worldwide, the computed tomography image based AI diagnostic system has become a sustainable solution for early diagnosis. However, the model-wise vulnerability under adversarial perturbation hinders its deployment in practical situation. The existing adversarial training strategies are difficult to generalized into medical imaging field challenged by complex medical texture features. To overcome this challenge, we propose a Contour Attention Preserving (CAP) method based on lung cavity edge extraction. The contour prior features are injected to attention layer via a parameter regularization and we optimize the robust empirical risk with hybrid distance metric. We then introduce a new cross-nation CT scan dataset to evaluate the generalization capability of the adversarial robustness under distribution shift. Experimental results indicate that the proposed method achieves state-of-the-art performance in multiple adversarial defense and generalization tasks. The code and dataset are available at https://github.com/Quinn777/CAP.

PDF Details DOI

ICLR Conference 2022 Conference Paper

Nonlinear ICA Using Volume-Preserving Transformations

Xiaojiang Yang
Yi Wang
Jiacheng Sun
Xing Zhang
Shifeng Zhang
Zhenguo Li
Junchi Yan

Nonlinear ICA is a fundamental problem in machine learning, aiming to identify the underlying independent components (sources) from data which is assumed to be a nonlinear function (mixing function) of these sources. Recent works prove that if the sources have some particular structures (e.g. temporal structure), they are theoretically identifiable even if the mixing function is arbitrary. However, in many cases such restrictions on the sources are difficult to satisfy or even verify, hence it inhibits the applicability of the proposed methods. Different from these works, we propose a general framework for nonlinear ICA, in which the mixing function is assumed to be a volume-preserving transformation, and meanwhile the conditions on the sources can be much looser. We provide an insightful proof of the identifiability of the proposed framework. We implement the framework by volume-preserving Flow-based models, and verify our theory by experiments on artificial data and synthesized images. Moreover, results on real-world images indicate that our framework can disentangle interpretable features.

Details

NeurIPS Conference 2021 Conference Paper

On Effective Scheduling of Model-based Reinforcement Learning

Hang Lai
Jian Shen
Weinan Zhang
Yimin Huang
Xing Zhang
Ruiming Tang
Yong Yu
Zhenguo Li

Model-based reinforcement learning has attracted wide attention due to its superior sample efficiency. Despite its impressive success so far, it is still unclear how to appropriately schedule the important hyperparameters to achieve adequate performance, such as the real data ratio for policy optimization in Dyna-style model-based algorithms. In this paper, we first theoretically analyze the role of real data in policy training, which suggests that gradually increasing the ratio of real data yields better performance. Inspired by the analysis, we propose a framework named AutoMBPO to automatically schedule the real data ratio as well as other hyperparameters in training model-based policy optimization (MBPO) algorithm, a representative running case of model-based methods. On several continuous control tasks, the MBPO instance trained with hyperparameters scheduled by AutoMBPO can significantly surpass the original one, and the real data ratio schedule found by AutoMBPO shows consistency with our theoretical analysis.

PDF Details

TCS Journal 2019 Journal Article

Hybrid fault diagnosis capability analysis of hypercubes under the PMC model and MM⁎ model

Qiang Zhu
Lili Li
Sanyang Liu
Xing Zhang

System level diagnosis is an important approach for the fault diagnosis of multiprocessor systems. In system level diagnosis, diagnosability is an important measure of the diagnosis capability of interconnection networks. But as a measure, diagnosability can not reflect the diagnosis capability of multiprocessor systems to link faults which may occur in real circumstances. In this paper, we propose the definition of h-edge tolerable diagnosability to better measure the diagnosis capability of interconnection networks under hybrid fault circumstances. The h-edge tolerable diagnosability of a multiprocessor system G is the maximum number of faulty nodes that the system can guarantee to locate when the number of faulty edges does not exceed h, denoted by t h e ( G ). The PMC model and MM model are the two most widely studied diagnosis models for the system level diagnosis of multiprocessor systems. The hypercubes are the most well-known interconnection networks. In this paper, the h-edge tolerable diagnosability of n-dimensional hypercube under the PMC and MM ⁎ model is determined as follows: t h e ( Q n ) = n − h, where 1 ≤ h < n, n ≥ 4.

Details DOI

IJCAI Conference 2019 Conference Paper

Unified Embedding Model over Heterogeneous Information Network for Personalized Recommendation

Zekai Wang
Hongzhi Liu
Yingpeng Du
Zhonghai Wu
Xing Zhang

Most of heterogeneous information network (HIN) based recommendation models are based on the user and item modeling with meta-paths. However, they always model users and items in isolation under each meta-path, which may lead to information extraction misled. In addition, they only consider structural features of HINs when modeling users and items during exploring HINs, which may lead to useful information for recommendation lost irreversibly. To address these problems, we propose a HIN based unified embedding model for recommendation, called HueRec. We assume there exist some common characteristics under different meta-paths for each user or item, and use data from all meta-paths to learn unified users’ and items’ representations. So the interrelation between meta-paths are utilized to alleviate the problems of data sparsity and noises on one meta-path. Different from existing models which first explore HINs then make recommendations, we combine these two parts into an end-to-end model to avoid useful information lost in initial phases. In addition, we embed all users, items and meta-paths into related latent spaces. Therefore, we can measure users’ preferences on meta-paths to improve the performances of personalized recommendation. Extensive experiments show HueRec consistently outperforms state-of-the-art methods.

PDF Details