Author name cluster

Xiangru Jian

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

EAAI Journal 2026 Journal Article

Spatio-temporal traffic accidents detection via graph based generative adversarial network

Lyuyi Zhu
Qixin Zhang
Xiangru Jian
Yu Yang
Lishuai Li

Due to urbanization and economic growth, traffic accidents have become a severe social problem. With the development of intelligent transportation systems and Internet of Things devices, detecting traffic accidents from big data is becoming an increasingly important trend for the future. However, there are several main challenges for accident detection. Firstly, traffic data is complex due to its spatial and temporal correlations. Secondly, traffic accidents are spatially and temporally dispersed, making them challenging to capture. Additionally, the high cost of labeling presents a significant obstacle, leading to a scarcity of available labels. Thirdly, unsupervised anomaly detection necessitates the approximation of normal samples, posing a challenge in approximating time series data in high-dimensional distributions collected from Internet of Things devices. To address these problems, we propose a novel spatio-temporal graph generative adversarial network framework, comprising a discriminator and a generator. The discriminator aims to identify fake and true samples by learning the representation of each input and its spatio-temporal context. The generator aims to generate fake data from the spatio-temporal context and fool the discriminator. Through adversarial training, the model can identify anomaly samples. We validate the performance of the proposed model on two real-world traffic accident datasets. The experimental results demonstrate that our model surpasses the baselines, thereby showcasing its effectiveness. Furthermore, a case study is conducted to analyze the characteristics and potential impact of the traffic accident, providing valuable insights for the improvement of this field and future research.

Details DOI

NeurIPS Conference 2025 Conference Paper

AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding

Ahmed Masry
Juan Rodriguez
Tianyu Zhang
Suyuchen Wang
Chao Wang
Aarash Feizi
Akshay Kalkunte Suresh
Abhay Puri

Aligning visual features with language embeddings is a key challenge in vision-language models (VLMs). The performance of such models hinges on having a good connector that maps visual features generated by a vision encoder to a shared embedding space with the LLM while preserving semantic similarity. Existing connectors, such as multilayer perceptrons (MLPs), lack inductive bias to constrain visual features within the linguistic structure of the LLM’s embedding space, making them data-hungry and prone to cross-modal misalignment. In this work, we propose a novel vision-text alignment method, AlignVLM, that maps visual features to a weighted average of LLM text embeddings. Our approach leverages the linguistic priors encoded by the LLM to ensure that visual features are mapped to regions of the space that the LLM can effectively interpret. AlignVLM is particularly effective for document understanding tasks, where visual and textual modalities are highly correlated. Our extensive experiments show that AlignVLM achieves state-of-the-art performance compared to prior alignment methods, with larger gains on document understanding and under low-resource setups. We provide further analysis demonstrating its efficiency and robustness to noise.

PDF Details

ICLR Conference 2025 Conference Paper

BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks

Juan A. Rodríguez
Xiangru Jian
Siba Smarak Panigrahi
Tianyu Zhang
Aarash Feizi
Abhay Puri
Akshay Kalkunte Suresh
François Savard

Multimodal AI has the potential to significantly enhance document-understanding tasks, such as processing receipts, understanding workflows, extracting data from documents, and summarizing reports. Code generation tasks that require long-structured outputs can also be enhanced by multimodality. Despite this, their use in commercial applications is often limited due to limited access to relevant training data and restrictive licensing, which hinders open access. To address these limitations, we introduce BigDocs-7.5M, a high-quality, open-access dataset comprising 7.5 million multimodal documents across 30 tasks. We use an efficient data curation process to ensure that our data is high quality and license-permissive. Our process emphasizes accountability, responsibility, and transparency through filtering rules, traceable metadata, and careful content analysis. Additionally, we introduce BigDocs-Bench,, a benchmark suite with 10 novel tasks where we carefully create datasets that reflect real-world use cases involving reasoning over Graphical User Interfaces (GUI) and code generation from images. Our experiments show that training with BigDocs-Bench, improves average performance up to 25.8% over closed-source GPT-4o in document reasoning and structured output tasks such as Screenshot2HTML or Image2Latex generation. Finally, human evaluations revealed that participants preferred the outputs from models trained with BigDocs over those from GPT-4o. This suggests that BigDocs can help both academics and the open-source community utilize and improve AI tools to enhance multimodal capabilities and document reasoning.

Details

EAAI Journal 2025 Journal Article

Graph convolutional network for traffic incidents duration classification

Lyuyi Zhu
Qixin Zhang
Xiangru Jian
Yu Yang

Traffic incidents are a primary cause of severe congestion in urban areas, making accurate forecasting of incident duration essential for effective traffic management systems. However, the inherent uncertainty associated with incidents presents significant challenges in predicting their durations. In this paper, we propose a novel deep neural network model for predicting and classifying traffic incident durations. To capture the dynamic nature of incidents, the model learns from time series data on traffic flow, speed, and occupancy. Additionally, it employs a graph neural network architecture to model the spatial relationships between sensors, while also accounting for factors such as time and incident type. By training the model with cross-entropy loss, we enable it to predict whether an incident’s duration will be long or short. Experimental results demonstrate that our model outperforms existing baselines, demonstrating the effectiveness of our proposed approach. Furthermore, we conduct a case study to visualize the impact of incidents and further validate the model’s predictive capability.

Details DOI

NeurIPS Conference 2025 Conference Paper

Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers

Wei Pang
Kevin Qinghong Lin
Xiangru Jian
Xi He
Philip Torr

Academic poster generation is a crucial yet challenging task in scientific communication, requiring the compression of long-context interleaved documents into a single, visually coherent page. To address this challenge, we introduce Paper2Poster, the first benchmark and metric suite for poster generation, which pairs recent conference papers with author-designed posters and evaluates outputs on (i) Visual Quality—semantic alignment with human posters, (ii) Textual Coherence—language fluency, (iii) Holistic Assessment—six fine-grained aesthetic and informational criteria scored by a VLM-as-judge, and notably (iv) PaperQuiz—the poster’s ability to convey core paper content as measured by VLMs answering generated quizzes. Building on this benchmark, we propose PosterAgent, a top‐down, visual‐in‐the‐loop multi‐agent pipeline: the (a) Parser distills the paper into a structured asset library; the (b) Planner aligns text–visual pairs into a binary‐tree layout that preserves reading order and spatial balance; and the (c) Painter–Commenter loop refines each panel by executing rendering code and using VLM feedback to eliminate overflow and ensure alignment. In our comprehensive evaluation, we find that GPT‐4o outputs—though visually appealing at first glance—often exhibit noisy text and poor PaperQuiz scores; We find that reader engagement is the primary aesthetic bottleneck, as human‐designed posters rely largely on visual semantics to convey meaning. Our fully open‐source Paper2Poster pipeline outperforms GPT‐4o–based systems across nearly all metrics while consuming 87 \% fewer tokens. These findings chart clear directions for the next generation of fully automated poster‐generation models.

PDF Details

TMLR Journal 2025 Journal Article

Rethinking Spectral Augmentation for Contrast-based Graph Self-Supervised Learning

Xiangru Jian
Xinjian Zhao
Wei Pang
Chaolong Ying
Yimu Wang
Yaoyao Xu
Tianshu Yu

The recent surge in contrast-based graph self-supervised learning has prominently featured an intensified exploration of spectral cues. Spectral augmentation, which involves modifying a graph's spectral properties such as eigenvalues or eigenvectors, is widely believed to enhance model performance. However, an intriguing paradox emerges, as methods grounded in seemingly conflicting assumptions regarding the spectral domain demonstrate notable enhancements in learning performance. Through extensive empirical studies, we find that simple edge perturbations - random edge dropping for node-level and random edge adding for graph-level self-supervised learning - consistently yield comparable or superior performance while being significantly more computationally efficient. This suggests that the computational overhead of sophisticated spectral augmentations may not justify their practical benefits. Our theoretical analysis of the InfoNCE loss bounds for shallow GNNs further supports this observation. The proposed insights represent a significant leap forward in the field, potentially refining the understanding and implementation of graph self-supervised learning.

PDF Details

NeurIPS Conference 2025 Conference Paper

The Underappreciated Power of Vision Models for Graph Structural Understanding

Xinjian Zhao
Wei Pang
Zhongkai Xue
Xiangru Jian
Lei Zhang
Yaoyao Xu
Xiaozhuang Song
Shu Wu

Graph Neural Networks operate through bottom-up message-passing, fundamentally differing from human visual perception, which intuitively captures global structures first. We investigate the underappreciated potential of vision models for graph understanding, finding they achieve performance comparable to GNNs on established benchmarks while exhibiting distinctly different learning patterns. These divergent behaviors, combined with limitations of existing benchmarks that conflate domain features with topological understanding, motivate our introduction of GraphAbstract. This benchmark evaluates models' ability to perceive global graph properties as humans do: recognizing organizational archetypes, detecting symmetry, sensing connectivity strength, and identifying critical elements. Our results reveal that vision models significantly outperform GNNs on tasks requiring holistic structural understanding and maintain generalizability across varying graph scales, while GNNs struggle with global pattern abstraction and degrade with increasing graph size. This work demonstrates that vision models possess remarkable yet underutilized capabilities for graph structural understanding, particularly for problems requiring global topological awareness and scale-invariant reasoning. These findings open new avenues to leverage this underappreciated potential for developing more effective graph foundation models for tasks dominated by holistic pattern recognition.

PDF Details

ICML Conference 2025 Conference Paper

UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction

Shravan Nayak
Xiangru Jian
Kevin Qinghong Lin
Juan A. Rodríguez
Montek Kalsi
Nicolas Chapados
M. Tamer Özsu
Aishwarya Agrawal

Autonomous agents that navigate Graphical User Interfaces (GUIs) to automate tasks like document editing and file management can greatly enhance computer workflows. While existing research focuses on online settings, desktop environments, critical for many professional and everyday tasks, remain underexplored due to data collection challenges and licensing issues. We introduce UI-Vision, the first comprehensive, license-permissive benchmark for offline, fine-grained evaluation of computer use agents in real-world desktop environments. Unlike online benchmarks, UI-Vision provides: (i) dense, high-quality annotations of human demonstrations, including bounding boxes, UI labels, and action trajectories (clicks, drags, and keyboard inputs) across 83 software applications, and (ii) three fine-to-coarse grained tasks—Element Grounding, Layout Grounding, and Action Prediction—with well-defined metrics to rigorously evaluate agents’ performance in desktop environments. Our evaluation reveals critical limitations in state-of-the-art models like UI-TARS-72B, including issues with understanding professional software, spatial reasoning, and complex actions like drag-and-drop. These findings highlight the challenges in developing fully autonomous computer-use agents. With UI-Vision, we aim to advance the development of more capable agents for real-world desktop tasks.

Details