Author name cluster

Ao Qu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

ICLR Conference 2025 Conference Paper

IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning

Vindula Jayawardana
Baptiste Freydt
Ao Qu
Cameron Hickert
Zhongxia Yan 0001
Cathy Wu 0002

Despite the popularity of multi-agent reinforcement learning (RL) in simulated and two-player applications, its success in messy real-world applications has been limited. A key challenge lies in its generalizability across problem variations, a common necessity for many real-world problems. Contextual reinforcement learning (CRL) formalizes learning policies that generalize across problem variations. However, the lack of standardized benchmarks for multi-agent CRL has hindered progress in the field. Such benchmarks are desired to be based on real-world applications to naturally capture the many open challenges of real-world problems that affect generalization. To bridge this gap, we propose IntersectionZoo, a comprehensive benchmark suite for multi-agent CRL through the real-world application of cooperative eco-driving in urban road networks. The task of cooperative eco-driving is to control a fleet of vehicles to reduce fleet-level vehicular emissions. By grounding IntersectionZoo in a real-world application, we naturally capture real-world problem characteristics, such as partial observability and multiple competing objectives. IntersectionZoo is built on data-informed simulations of 16,334 signalized intersections derived from 10 major US cities, modeled in an open-source industry-grade microscopic traffic simulator. By modeling factors affecting vehicular exhaust emissions (e.g., temperature, road conditions, travel demand), IntersectionZoo provides one million data-driven traffic scenarios. Using these traffic scenarios, we benchmark popular multi-agent RL and human-like driving algorithms and demonstrate that the popular multi-agent RL algorithms struggle to generalize in CRL settings.

Details

NeurIPS Conference 2025 Conference Paper

Simulating Society Requires Simulating Thought

Chance Jiajie Li
Jiayi Wu
Zhenze MO
Ao Qu
Yuhan Tang
Kaiya Zhao
Yulu Gan
Jie Fan

Simulating society with large language models (LLMs), we argue, requires more than generating plausible behavior; it demands cognitively grounded reasoning that is structured, revisable, and traceable. LLM-based agents are increasingly used to emulate individual and group behavior, primarily through prompting and supervised fine-tuning. Yet current simulations remain grounded in a behaviorist “demographics in, behavior out” paradigm, focusing on surface-level plausibility. As a result, they often lack internal coherence, causal reasoning, and belief traceability—making them unreliable for modeling how people reason, deliberate, and respond to interventions. To address this, we present a conceptual modeling paradigm, Generative Minds (GenMinds), which draws from cognitive science to support structured belief representations in generative agents. To evaluate such agents, we introduce the RECAP (REconstructing CAusal Paths) framework, a benchmark designed to assess reasoning fidelity via causal traceability, demographic grounding, and intervention consistency. These contributions advance a broader shift: from surface-level mimicry to generative agents that simulate thought—not just language—for social simulations.

PDF Details

TIST Journal 2023 Journal Article

Adversarial Attacks on Deep Reinforcement Learning-based Traffic Signal Control Systems with Colluding Vehicles

Ao Qu
Yihong Tang
Wei Ma

The rapid advancements of Internet of Things (IoT) and Artificial Intelligence (AI) have catalyzed the development of adaptive traffic control systems (ATCS) for smart cities. In particular, deep reinforcement learning (DRL) models produce state-of-the-art performance and have great potential for practical applications. In the existing DRL-based ATCS, the controlled signals collect traffic state information from nearby vehicles, and then optimal actions (e.g., switching phases) can be determined based on the collected information. The DRL models fully “trust” that vehicles are sending the true information to the traffic signals, making the ATCS vulnerable to adversarial attacks with falsified information. In view of this, this article first time formulates a novel task in which a group of vehicles can cooperatively send falsified information to “cheat” DRL-based ATCS in order to save their total travel time. To solve the proposed task, we develop CollusionVeh, a generic and effective vehicle-colluding framework composed of a road situation encoder, a vehicle interpreter, and a communication mechanism. We employ our framework to attack established DRL-based ATCS and demonstrate that the total travel time for the colluding vehicles can be significantly reduced with a reasonable number of learning episodes, and the colluding effect will decrease if the number of colluding vehicles increases. Additionally, insights and suggestions for the real-world deployment of DRL-based ATCS are provided. The research outcomes could help improve the reliability and robustness of the ATCS and better protect the smart mobility systems.

Details DOI

TIST Journal 2022 Journal Article

Detecting Extreme Traffic Events Via a Context Augmented Graph Autoencoder

Yue Hu
Ao Qu
Dan Work

Accurate and timely detection of large events on urban transportation networks enables informed mobility management. This work tackles the problem of extreme event detection on large-scale transportation networks using origin-destination mobility data, which is now widely available. Such data is highly structured in time and space, but high dimensional and sparse. Current multivariate time series anomaly detection methods cannot fully address these challenges. To exploit the structure of mobility data, we formulate the event detection problem in a novel way, as detecting anomalies in a set of time-dependent directed weighted graphs. We further propose a Context augmented Graph Autoencoder (Con-GAE) model to solve the problem, which leverages graph embedding and context embedding techniques to capture the spatial and temporal patterns. Con-GAE adopts an autoencoder framework and detects anomalies via semi-supervised learning. The performance of the method is assessed on several city-scale travel-time datasets from Uber Movement, New York taxis, and Chicago taxis and compared to state-of-the-art approaches. The proposed Con-GAE can achieve an improvement in the area under the curve score as large as 0.15 over the second best method. We also discuss real-world traffic anomalies detected by Con-GAE.

Details DOI