Author name cluster

Xuan Yang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

AAAI Conference 2025 Conference Paper

LLM-RG4: Flexible and Factual Radiology Report Generation Across Diverse Input Contexts

Zhuhao Wang
Yihua Sun
Zihan Li
Xuan Yang
Fang Chen
Hongen Liao

Drafting radiology reports is a complex task requiring flexibility, where radiologists tail content to available information and particular clinical demands. However, most current radiology report generation (RRG) models are constrained to a fixed task paradigm, such as predicting the full ''finding'' section from a single image, inherently involving a mismatch between inputs and outputs. The trained models lack the flexibility for diverse inputs and could generate harmful, input-agnostic hallucinations. To bridge the gap between current RRG models and the clinical demands in practice, we first develop a data generation pipeline to create a new MIMIC-RG4 dataset, which considers four common radiology report drafting scenarios and has perfectly corresponded input and output. Secondly, we propose a novel large language model (LLM) based RRG framework, namely LLM-RG4, which utilizes LLM's flexible instruction-following capabilities and extensive general knowledge. We further develop an adaptive token fusion module that offers flexibility to handle diverse scenarios with different input combinations, while minimizing the additional computational burden associated with increased input volumes. Besides, we propose a token-level loss weighting strategy to direct the model's attention towards positive and uncertain descriptions. Experimental results demonstrate that LLM-RG4 achieves state-of-the-art performance in both clinical efficiency and natural language generation on the MIMIC-RG4 and MIMIC-CXR datasets. We quantitatively demonstrate that our model has minimal input-agnostic hallucinations, whereas current open-source models commonly suffer from this problem.

PDF Details DOI

ICML Conference 2025 Conference Paper

SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models

Jiawei Zhang 0013
Xuan Yang
Taiqi Wang
Yu Yao
Aleksandr Petiushko
Bo Li 0026

Traditional autonomous driving systems often struggle to connect high-level reasoning with low-level control, leading to suboptimal and sometimes unsafe behaviors. Recent advances in multimodal large language models (MLLMs), which process both visual and textual data, offer an opportunity to unify perception and reasoning. However, effectively embedding precise safety knowledge into MLLMs for autonomous driving remains a significant challenge. To address this, we propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge. First, we introduce a Position-Dependent Cross-Entropy (PDCE) loss to improve low-level control signal predictions when values are represented as text. Second, to explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic (e. g. , "red light =$>$ stop") and embeds them into a probabilistic graphical model (e. g. , Markov Logic Network) to verify predicted actions using recognized environmental attributes. Additionally, our Multimodal Retrieval-Augmented Generation (RAG) model leverages video, control signals, and environmental attributes to learn from past driving experiences. Integrating PDCE, MLN, and Multimodal RAG, SafeAuto outperforms existing baselines across multiple datasets, enabling more accurate, reliable, and safer autonomous driving. The code is available at https: //github. com/AI-secure/SafeAuto.

Details

EAAI Journal 2024 Journal Article

A novel uncertainty modeling method in complex evidence theory for decision making

Xuan Yang
Fuyuan Xiao

The Dempster-Shafer (D-S) evidence theory, renowned for its efficacy in processing uncertain information, has garnered substantial attention in recent decades. A generalized D-S evidence theory called complex evidence theory (CET), is tailored to model and handle uncertainty on the complex plane. Within this context, generating complex basic belief assignments (CBBA) in CET stands as a critical challenge to improve the decision level for CET-based information fusion. This paper proposes an innovative method for generating CBBA to model uncertainty on the complex plane. Initially, a complex attribute membership function is introduced to construct the model. Subsequently, the CBBA is generated by inputting data into this model. Based on this CBBA generation method, a new decision-making algorithm is presented. To validate the effectiveness of the proposed CBBA generation method and decision-making algorithm, they are applied in a classification application. Comparative analysis demonstrates the superior performance of the proposed algorithm in CBBA generation compared to other methods.

Details DOI

EAAI Journal 2024 Journal Article

An improved density peaks clustering algorithm based on the generalized neighbors similarity

Xuan Yang
Fuyuan Xiao

Density peaks clustering (DPC) algorithm reported in Science is a novel and efficient clustering method which has attracted great attention for its simplicity and practicability. Although it has shown promising results in some applications, there still exist some certain disadvantages. For example, the calculation method of local density without taking into account the impact of the surrounding areas may cause the wrong cluster centers selection results. In spite of the simple data points allocation strategy, the allocation strategy may cause the serial incorrect cluster results. Given these disadvantages of DPC algorithm, we propose a nearest neighbors similarity based clustering method which is called generalized neighbors similarity based clustering by fast search and find of density peaks (abbreviated as GNS-DPC). Considering the data points’ K-nearest-neighbor information, we give a generalized neighbors similarity measurement between data points and present a new definition of local density and relative distance. In the data points allocation stage, this GNS-DPC also takes advantage of the nearest neighbors’information of a data point. The allocation process consists of several steps, which can solve the serial incorrect cluster results problem. The experimental results suggest that our method can correctly obtain the cluster centers and recognize clusters with higher accuracy.

Details DOI

TMLR Journal 2024 Journal Article

VideoGLUE: Video General Understanding Evaluation of Foundation Models

Liangzhe Yuan
Nitesh Bharadwaj Gundavarapu
Long Zhao
Hao Zhou
Yin Cui
Lu Jiang
Xuan Yang
Menglin Jia

We evaluate the video understanding capabilities of existing foundation models (FMs) using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition,temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring an FM for downstream tasks. Furthermore,we jointly profile FMs’ efficacy and efficiency when adapting to general video understanding tasks using cost measurements during both training and inference. Our main findings areas follows. First, task-specialized models significantly outperform the seven FMs studied in this work, in sharp contrast to what FMs have achieved in natural language and image understanding. Second, video-native FMs, whose pretraining data mainly contains the video modality, are generally better than image-native FMs in classifying motion-rich videos,localizing actions in time, and understanding a video of more than one action. Third, the video-native FMs can perform well on video tasks under light adaptations to downstream tasks (e.g., freezing the FM backbones), while image-native FMs win in full end-to-end finetuning. The first two observations reveal the need and tremendous opportunities to conduct research on video-focused FMs, and the last confirms that both tasks and adaptation methods matter when it comes to the evaluation of FMs. Our code is released under: https://github.com/tensorflow/models/tree/master/official/projects/videoglue

PDF Details

ICML Conference 2024 Conference Paper

VideoPoet: A Large Language Model for Zero-Shot Video Generation

Dan Kondratyuk
Lijun Yu
Xiuye Gu
José Lezama
Jonathan Huang
Grant Schindler
Rachel Hornung
Vighnesh Birodkar

We present VideoPoet, a language model capable of synthesizing high-quality video from a large variety of conditioning signals. VideoPoet employs a decoder-only transformer architecture that processes multimodal inputs – including images, videos, text, and audio. The training protocol follows that of Large Language Models (LLMs), consisting of two stages: pretraining and task-specific adaptation. During pretraining, VideoPoet incorporates a mixture of multimodal generative objectives within an autoregressive Transformer framework. The pretrained LLM serves as a foundation that can be adapted for a range of video generation tasks. We present empirical results demonstrating the model’s state-of-the-art capabilities in zero-shot video generation, specifically highlighting the ability to generate high-fidelity motions. Project page: http: //sites. research. google/videopoet/

Details

NeurIPS Conference 2023 Conference Paper

DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model

Xiuye Gu
Yin Cui
Jonathan Huang
Abdullah Rashwan
Xuan Yang
Xingyi Zhou
Golnaz Ghiasi
Weicheng Kuo

Observing the close relationship among panoptic, semantic and instance segmentation tasks, we propose to train a universal multi-dataset multi-task segmentation model: DaTaSeg. We use a shared representation (mask proposals with class predictions) for all tasks. To tackle task discrepancy, we adopt different merge operations and post-processing for different tasks. We also leverage weak-supervision, allowing our segmentation model to benefit from cheaper bounding box annotations. To share knowledge across datasets, we use text embeddings from the same semantic embedding space as classifiers and share all network parameters among datasets. We train DaTaSeg on ADE semantic, COCO panoptic, and Objects365 detection datasets. DaTaSeg improves performance on all datasets, especially small-scale datasets, achieving 54. 0 mIoU on ADE semantic and 53. 5 PQ on COCO panoptic. DaTaSeg also enables weakly-supervised knowledge transfer on ADE panoptic and Objects365 instance segmentation. Experiments show DaTaSeg scales with the number of training datasets and enables open-vocabulary segmentation through direct transfer. In addition, we annotate an Objects365 instance segmentation set of 1, 000 images and release it as a public evaluation benchmark on https: //laoreja. github. io/dataseg.

PDF Details

AAAI Conference 2023 Conference Paper

DropMessage: Unifying Random Dropping for Graph Neural Networks

Taoran Fang
Zhiqing Xiao
Chunping Wang
Jiarong Xu
Xuan Yang
Yang Yang

Graph Neural Networks (GNNs) are powerful tools for graph representation learning. Despite their rapid development, GNNs also face some challenges, such as over-fitting, over-smoothing, and non-robustness. Previous works indicate that these problems can be alleviated by random dropping methods, which integrate augmented data into models by randomly masking parts of the input. However, some open problems of random dropping on GNNs remain to be solved. First, it is challenging to find a universal method that are suitable for all cases considering the divergence of different datasets and models. Second, augmented data introduced to GNNs causes the incomplete coverage of parameters and unstable training process. Third, there is no theoretical analysis on the effectiveness of random dropping methods on GNNs. In this paper, we propose a novel random dropping method called DropMessage, which performs dropping operations directly on the propagated messages during the message-passing process. More importantly, we find that DropMessage provides a unified framework for most existing random dropping methods, based on which we give theoretical analysis of their effectiveness. Furthermore, we elaborate the superiority of DropMessage: it stabilizes the training process by reducing sample variance; it keeps information diversity from the perspective of information theory, enabling it become a theoretical upper bound of other methods. To evaluate our proposed method, we conduct experiments that aims for multiple tasks on five public datasets and two industrial datasets with various backbone models. The experimental results show that DropMessage has the advantages of both effectiveness and generalization, and can significantly alleviate the problems mentioned above. A detailed version with full appendix can be found on arXiv: https://arxiv.org/abs/2204.10037.

PDF Details DOI