Author name cluster

Wentian Zhao

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

1 author row

NeurIPS Conference 2025 Conference Paper

Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference

Jiayi Yuan
Hao Li
Xinheng Ding
Wenya Xie
Yu-Jhe Li
Wentian Zhao
Kun Wan
Jing Shi

Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are both accurate and reproducible. We demonstrate that the reproducibility of LLM performance is fragile: changing system configuration, such as evaluation batch size, GPU count, and GPU version, can introduce significant differences in the generated responses. This issue is especially pronounced in reasoning models, where minor rounding differences in early tokens can cascade into divergent chains of thought, ultimately affecting accuracy. For instance, under bfloat16 precision with greedy decoding, a reasoning model like DeepSeek-R1-Distill-Qwen-7B can exhibit up to 9\% variation in accuracy and 9, 000 tokens difference in response length due to differences in GPU count, type, and evaluation batch size. We trace the root cause of this variability to the non-associative nature of floating-point arithmetic under limited numerical precision. This work presents the first systematic investigation into how numerical precision affects reproducibility in LLM inference. Through carefully controlled experiments across various hardware, software, and precision settings, we quantify when and how model outputs diverge. Our analysis reveals that floating-point precision—while critical for reproducibility—is often neglected in evaluation practices. Inspired by this, we develop a lightweight inference pipeline, dubbed LayerCast, that stores weights in 16-bit precision but performs all computations in FP32, balancing memory efficiency with numerical stability. Code is available at https: //github. com/nanomaoli/llm_reproducibility.

PDF Details

AAAI Conference 2024 Conference Paper

Relational Distant Supervision for Image Captioning without Image-Text Pairs

Yayun Qi
Wentian Zhao
Xinxiao Wu

Unsupervised image captioning aims to generate descriptions of images without relying on any image-sentence pairs for training. Most existing works use detected visual objects or concepts as bridge to connect images and texts. Considering that the relationship between objects carries more information, we use the object relationship as a more accurate connection between images and texts. In this paper, we adapt the idea of distant supervision that extracts the knowledge about object relationships from an external corpus and imparts them to images to facilitate inferring visual object relationships, without introducing any extra pre-trained relationship detectors. Based on these learned informative relationships, we construct pseudo image-sentence pairs for captioning model training. Specifically, our method consists of three modules: (1) a relationship learning module that learns to infer relationships from images under the distant supervision; (2) a relationship-to-sentence module that transforms the inferred relationships into sentences to generate pseudo image-sentence pairs; (3) an image captioning module that is trained by using the generated image-sentence pairs. Promising results on three datasets show that our method outperforms the state-of-the-art methods of unsupervised image captioning.

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Multi-modal Dependency Tree for Video Captioning

Wentian Zhao
Xinxiao Wu
Jiebo Luo

Generating fluent and relevant language to describe visual content is critical for the video captioning task. Many existing methods generate captions using sequence models that predict words in a left-to-right order. In this paper, we investigate a graph-structured model for caption generation by explicitly modeling the hierarchical structure in the sentences to further improve the fluency and relevance of sentences. To this end, we propose a novel video captioning method that generates a sentence by first constructing a multi-modal dependency tree and then traversing the constructed tree, where the syntactic structure and semantic relationship in the sentence are represented by the tree topology. To take full advantage of the information from both vision and language, both the visual and textual representation features are encoded into each tree node. Different from existing dependency parsing methods that generate uni-modal dependency trees for language understanding, our method construct s multi-modal dependency trees for language generation of images and videos. We also propose a tree-structured reinforcement learning algorithm to effectively optimize the captioning model where a novel reward is designed by evaluating the semantic consistency between the generated sub-tree and the ground-truth tree. Extensive experiments on several video captioning datasets demonstrate the effectiveness of the proposed method.

PDF Details

AAAI Conference 2020 Conference Paper

MemCap: Memorizing Style Knowledge for Image Captioning

Wentian Zhao
Xinxiao Wu
Xiaoxun Zhang

Generating stylized captions for images is a challenging task since it requires not only describing the content of the image accurately but also expressing the desired linguistic style appropriately. In this paper, we propose MemCap, a novel stylized image captioning method that explicitly encodes the knowledge about linguistic styles with memory mechanism. Rather than relying heavily on a language model to capture style factors in existing methods, our method resorts to memorizing stylized elements learned from training corpus. Particularly, we design a memory module that comprises a set of embedding vectors for encoding style-related phrases in training corpus. To acquire the style-related phrases, we develop a sentence decomposing algorithm that splits a stylized sentence into a style-related part that reﬂects the linguistic style and a content-related part that contains the visual content. When generating captions, our MemCap ﬁrst extracts content-relevant style knowledge from the memory module via an attention mechanism and then incorporates the extracted knowledge into a language model. Extensive experiments on two stylized image captioning datasets (Senti- Cap and FlickrStyle10K) demonstrate the effectiveness of our method.

PDF Details

IJCAI Conference 2020 Conference Paper

Video Question Answering on Screencast Tutorials

Wentian Zhao
Seokhwan Kim
Ning Xu
Hailin Jin

This paper presents a new video question answering task on screencast tutorials. We introduce a dataset including question, answer and context triples from the tutorial videos for a software. Unlike other video question answering works, all the answers in our dataset are grounded to the domain knowledge base. An one-shot recognition algorithm is designed to extract the visual cues, which helps enhance the performance of video question answering. We also propose several baseline neural network architectures based on various aspects of video contexts from the dataset. The experimental results demonstrate that our proposed models significantly improve the question answering performances by incorporating multi-modal contexts and domain knowledge.

PDF Details DOI

IJCAI Conference 2019 Conference Paper

GAN-EM: GAN Based EM Learning Framework

Wentian Zhao
Shaojie Wang
Zhihuai Xie
Jing Shi
Chenliang Xu

Expectation maximization (EM) algorithm is to find maximum likelihood solution for models having latent variables. A typical example is Gaussian Mixture Model (GMM) which requires Gaussian assumption, however, natural images are highly non-Gaussian so that GMM cannot be applied to perform image clustering task on pixel space. To overcome such limitation, we propose a GAN based EM learning framework that can maximize the likelihood of images and estimate the latent variables. We call this model GAN-EM, which is a framework for image clustering, semi-supervised classification and dimensionality reduction. In M-step, we design a novel loss function for discriminator of GAN to perform maximum likelihood estimation (MLE) on data with soft class label assignments. Specifically, a conditional generator captures data distribution for K classes, and a discriminator tells whether a sample is real or fake for each class. Since our model is unsupervised, the class label of real data is regarded as latent variable, which is estimated by an additional network (E-net) in E-step. The proposed GAN-EM achieves state-of-the-art clustering and semi-supervised classification results on MNIST, SVHN and CelebA, as well as comparable quality of generated images to other recently developed generative models.

PDF Details

EAAI Journal 2016 Journal Article

Fault diagnosis network design for vehicle on-board equipments of high-speed railway: A deep learning approach

Jiateng Yin
Wentian Zhao

With the rapid development of high-speed railways (HSRs) throughout the world, the fault diagnosis systems of vehicle on-board equipments (VOBEs) for high speed trains have received increasing attention. Since the faults of VOBEs in HSRs are usually uncertain and complex, the current fault diagnosis methods are mainly based on manual judgement in real-world operations, which is generally inefficient and insecurity with the big rail traffic data. In this paper, we propose an automated diagnosis network of VOBE for high-speed train via a deep learning approach. First, we propose a mathematical model to formulate the fault diagnosis problem in HSRs, involving the definition of fault evidence vectors and reason vectors by analyzing the real-world fault data that are collected in Wuhan-Guangzhou high speed railway. Then, a deep belief network (DBN) and its training procedures are developed on the basis of Restricted Boltzmann Machine (RBM). Finally, the proposed diagnosis network is trained and validated with real-world data. Furthermore, we compare the DBN-based fault diagnosis network with k-nearest neighbor (KNN) and ANN-BP (artificial neural network with back propagations). The results indicate that, the developed DBN outperforms both KNN and ANN-BP, and improves the accuracy of fault diagnosis for VOBEs to 90–95% in HSRs.

Details DOI