Author name cluster

Ziyan Yang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

NeurIPS Conference 2025 Conference Paper

Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations

Jiaming Han
Hao Chen
Yang Zhao
Hanyu Wang
Qi Zhao
Ziyan Yang
Hao He
Xiangyu Yue

This paper presents a multimodal framework that attempts to unify visual understanding and generation within a shared discrete semantic representation. At its core is the Text-Aligned Tokenizer (TA-Tok), which converts images into discrete tokens using a text-aligned codebook projected from a large language model's (LLM) vocabulary. By integrating vision and text into a unified space with an expanded vocabulary, our multimodal LLM, Tar, enables cross-modal input and output through a shared interface, without the need for modality-specific designs. Additionally, we propose scale-adaptive encoding and decoding to balance efficiency and visual detail, along with a generative de-tokenizer to produce high-fidelity visual outputs. To address diverse decoding needs, we utilize two complementary de-tokenizers: a fast autoregressive model and a diffusion-based model. To enhance modality fusion, we investigate advanced pre-training tasks, demonstrating improvements in both visual understanding and generation. Experiments across benchmarks show that Tar matches or surpasses existing multimodal LLM methods, achieving faster convergence and greater training efficiency. All code, models, and data will be made publicly available.

PDF Details

TMLR Journal 2024 Journal Article

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

Dongruo Zhou
Jinghui Chen
Yuan Cao
Ziyan Yang
Quanquan Gu

Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly studied. In this paper, we provide a fine-grained convergence analysis for a general class of adaptive gradient methods including AMSGrad, RMSProp and AdaGrad. For smooth nonconvex functions, we prove that adaptive gradient methods in expectation converge to a first-order stationary point. Our convergence rate is better than existing results for adaptive gradient methods in terms of dimension. In addition, we also prove high probability bounds on the convergence rates of AMSGrad, RMSProp as well as AdaGrad, which have not been established before. Our analyses shed light on better understanding the mechanism behind adaptive gradient methods in optimizing nonconvex objectives.

PDF Details

YNICL Journal 2021 Journal Article

Impaired probabilistic reversal learning in anxiety: Evidence from behavioral and ERP findings

Lisheng Xia
Pengfei Xu
Ziyan Yang
Ruolei Gu
Dandan Zhang

BACKGROUND: Reversal learning reflects an individual's capacity to adapt to a dynamic environment with changing stimulus-reward contingencies. This study focuses on the potential influence of anxiety on reversal learning skills. METHODS: We asked 40 participants with a high level of trait anxiety (HTA) and 40 counterparts with a low anxiety level (LTA) to finish a probabilistic reversal learning task with event-related potential (ERP) recording, during which stimulus-reward contingencies are reversed after players have learned the optimal choice. RESULTS: We found that compared to their LTA counterparts, the HTA participants showed worse learning performance and were less likely to make lose-shift choices. The FRN amplitude might help interpret these behavioral results, which is suggested to be associated with punishment sensitivity and was positively correlated with the number of lose-shift in this study. Seeing that anxiety level predicted the FRN amplitude for lose-shift, we explain that anxious individuals' inflexible behavioral responses to losses are due to their impaired sensitivity to negative feedback. CONCLUSIONS: A higher level of anxiety is associated with weaker reversal learning performance, possibly because of abnormal sensitivity to negative outcomes. These findings have implications for the understanding of behavioral symptoms in anxiety.

Details DOI

IJCAI Conference 2020 Conference Paper

Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks

Jinghui Chen
Dongruo Zhou
Yiqi Tang
Ziyan Yang
Yuan Cao
Quanquan Gu

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the generalization gap of adaptive gradient methods an open problem. In this work, we show that adaptive gradient methods such as Adam, Amsgrad, are sometimes "over adapted". We design a new algorithm, called Partially adaptive momentum estimation method, which unifies the Adam/Amsgrad with SGD by introducing a partial adaptive parameter $p$, to achieve the best from both worlds. We also prove the convergence rate of our proposed algorithm to a stationary point in the stochastic nonconvex optimization setting. Experiments on standard benchmarks show that our proposed algorithm can maintain fast convergence rate as Adam/Amsgrad while generalizing as well as SGD in training deep neural networks. These results would suggest practitioners pick up adaptive gradient methods once again for faster training of deep neural networks.

PDF Details DOI