Author name cluster

Sunan Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

EAAI Journal 2026 Journal Article

A vehicle lateral control algorithm that directly performs safe self-learning in real-world environments

Ke Wu
Ping Lu
Sunan Zhang
Bocheng Liu
Xin Ye
Bo Hu

Lateral Control algorithms in autonomous vehicles that perform well in simulation environments often fail to guarantee performance and safety in real-world scenarios, primarily due to significant differences between the simulated and actual environments. While reinforcement learning (RL) enables vehicles to interact and learn in real-world environments to enhance their performance, ensuring safety during the learning process remains challenging. This paper takes into full consideration the errors between the simulation model and the real environment as well as the propagation of these errors during the prediction process. Through a learning-based Model Predictive Control (MPC) algorithm, the vehicle's lateral control is maintained safely during the online learning process, with the model and the final control performance being continuously refined using data generated from interactions with the environment. Additionally, known priors are utilized to guarantee the initial performance of the policy in real-world environments and accelerate the evolution of the algorithm. Simulation and experiment results show that the proposed algorithm can guarantee safety during the learning process with a high probability and achieve better performance after self-learning. In contrast to the traditional RL paradigm, which learns in a simulation environment before applying its knowledge in the real world, the proposed algorithm stands out by its capability to self-evolve directly within the real environment safely, making it an appealing option for enhancing various model-based methods that require continuous model refinement for further development.

Details DOI

EAAI Journal 2025 Journal Article

A knowledge-guided reinforcement learning method for lateral path tracking

Bo Hu
Sunan Zhang
Yuxiang Feng
Bingbing Li
Hao Sun
Mingyang Chen
Weichao Zhuang
Yi Zhang

Lateral Control algorithms in autonomous vehicles often necessitates an online fine-tuning procedure in the real world. While reinforcement learning (RL) enables vehicles to learn and improve the lateral control performance through repeated trial and error interactions with a dynamic environment, applying RL directly to safety-critical applications in real physical world is challenging because ensuring safety during the learning process remains difficult. To enable safe learning, a promising direction is to make use of previously gathered offline data, which is frequently accessible in engineering applications. In this context, this paper presents a set of knowledge-guided RL algorithms that can not only fully leverage the prior collected offline data without the need of a physics-based simulator, but also allow further online policy improvement in a smooth, safe and efficient manner. To evaluate the effectiveness of the proposed algorithms on a real controller, a hardware-in-the-loop and a miniature vehicle platform are built. Compared with the vanilla RL, behavior cloning and the existing controller, the proposed algorithms realize a closed-loop solution for lateral control problems from offline training to online fine-tuning, making it attractive for future similar RL-based controller to build upon.

Details DOI

EAAI Journal 2025 Journal Article

An uncertainty-aware safe-evolving reinforcement learning algorithm for decision-making and control in highway autonomous driving

Ping Lu
Sunan Zhang
Feihong Tan
Fulin Zhang
Yuxiang Feng
Bo Hu

Rule-based and optimization-based approaches face challenges in decision-making and control for autonomous vehicles (AVs) in dynamic and complex highway scenarios. In contrast, reinforcement learning (RL) offers a more flexible and adaptable data-driven solution by allowing AVs to learn optimal actions through interactions with the environment, without requiring predefined rules or explicit programming. However, in real-world highway environments characterized by uncertainty, RL algorithm encounter difficulties in ensuring stability and safety. To address these challenges, this paper proposes an uncertainty-aware safe-evolving RL algorithm that integrates internal stability, external stability, and provable safety mechanisms. The internal stability mechanism ensures consistent performance improvements with high probability during policy updates itself, while the external stability leverages a benchmark policy as a reference to ensure the current policy performs at least as well as, if not better than, the benchmark. Furthermore, an action projection mechanism and a mixed learning procedure are incorporated to make minimal modifications to the learned policy, ensuring safety while supporting stable learning from both safe and original actions. The results show that the proposed algorithm maintains stability and safety throughout the learning process, achieves final performance comparable to traditional RL methods, and delivers higher training efficiency in a complex dynamic highway scenario in simulation. This suggests that the algorithm offers a viable solution for self-evolving systems in uncertain real-world environments, where traditional approaches may struggle.

Details DOI