Author name cluster

Zhiyan Dong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

TAAS Journal 2026 Journal Article

Auto-Follower: A Person-Following System for Urban Ackermann Human–Machine Collaborative Robotics

Zhijian Li
Dongliang Kou
Yizhao Wang
Wei Li
Zhiyan Dong
Lihua Zhang

Industry 5.0 is emerging as the next phase of industrial evolution, emphasizing human-centric manufacturing through close human–robot collaboration and the deployment of intelligent autonomous systems. As a representative example of such autonomy, person-following robots are typically implemented on differential-drive or omnidirectional mobile bases. However, certain tasks require Ackermann-steered robots, which face unique challenges due to limited maneuverability and the complexity of urban environments, often leading to target loss or navigation into non-drivable areas. To address these issues, we propose Auto-Follower, a person-following framework with enhanced perception and navigation capabilities. Auto-Follower integrates a vision–LiDAR servo tracker that fuses camera images with LiDAR points from a motorized rotating sensor, enabling 360° target perception. Instead of relying on a global map, the system employs real-time LiDAR-based local mapping for efficient path planning. In addition, an Iterative Radius Points Search (IRPS) method is developed to identify obstacle-free navigation goals when the target enters non-drivable regions, ensuring safe and continuous following. The framework has been validated extensively in both laboratory and urban environments and demonstrates robust, reliable performance, with strong potential for adaptation to diverse real-world person-following applications.

Details DOI

ICRA Conference 2025 Conference Paper

Continuous Control of Diverse Skills in Quadruped Robots Without Complete Expert Datasets

Jiaxin Tu
Xiaoyi Wei
Yueqi Zhang
Taixian Hou
Xiaofei Gao
Zhiyan Dong
Peng Zhai
Lihua Zhang 0002

Learning diverse skills for quadruped robots presents significant challenges, such as mastering complex transitions between different skills and handling tasks of varying difficulty. Existing imitation learning methods, while successful, rely on expensive datasets to reproduce expert behaviors. Inspired by introspective learning, we propose Progressive Adversarial Self-Imitation Skill Transition (PASIST), a novel method that eliminates the need for complete expert datasets. PASIST autonomously explores and selects high-quality trajectories based on predefined target poses instead of demonstrations, leveraging the Generative Adversarial Self-Imitation Learning (GASIL) framework. To further enhance learning, We develop a skill selection module to mitigate mode collapse by balancing the weights of skills with varying levels of difficulty. Through these methods, PASIST is able to reproduce skills corresponding to the target pose while achieving smooth and natural transitions between them. Evaluations on both simulation platforms and the Solo 8 robot confirm the effectiveness of PASIST, offering an efficient alternative to expert-driven learning.

Details

ICRA Conference 2025 Conference Paper

Music-Driven Legged Robots: Synchronized Walking to Rhythmic Beats

Taixian Hou
Yueqi Zhang
Xiaoyi Wei
Zhiyan Dong
Jiafu Yi
Peng Zhai
Lihua Zhang 0002

We address the challenge of effectively controlling the locomotion of legged robots by incorporating precise frequency and phase characteristics, which is often ignored in locomotion policies that do not account for the periodic nature of walking. We propose a hierarchical architecture that integrates a low-level phase tracker, oscillators, and a high-level phase modulator. This controller allows quadruped robots to walk in a natural manner that is synchronized with external musical rhythms. Our method generates diverse gaits across different frequencies and achieves real-time synchronization with music in the physical world. This research establishes a foundational framework for enabling real-time execution of accurate rhythmic motions in legged robots. The video and code are available at https://music-walker.github.io/.

Details

IROS Conference 2025 Conference Paper

Robust Reinforcement Learning based on Momentum Adversarial Training

Li He
Hanchen Liu
Junru Sheng
Lihua Zhang
Zhiyan Dong

Reinforcement learning (RL) is a fundamental and pivotal algorithm in the advancement of autonomous intelligence, including Embodied Intelligence and Physical Intelligence. The performance of RL directly influences the quality and efficiency of a robot’s decision-making and execution during interactions with its environment. Moreover, the robustness of RL remains a critical challenge that needs to be addressed. A promising approach to enhancing robustness is adversarial reinforcement learning. However, the existing methods primarily focus on perturbations in the state space, while perturbations in the action space have been relatively underexplored. The action space in RL is as crucial as the state space in autonomous intelligence. Furthermore, action-space perturbations provide a more comprehensive evaluation of RL robustness. Therefore, it is necessary and valuable to investigate RL robustness under action-space perturbations for the development of autonomous intelligence. To this end, we propose an adversarial learning framework that employs momentum-based gradient descent to model perturbations in the action space, such as actuator disturbances. Furthermore, we introduce an improved optimization method that integrates historical gradient information into conventional Stochastic Gradient Descent (SGD). This approach enhances training stability and improves perturbation efficiency. The proposed method is evaluated through simulations in the MuJoCo environment and UAV control experiments in GymFC, demonstrating significant improvements in robustness and adaptability under action-space perturbations. Additionally, real-world UAV flight tests are conducted to further validate the effectiveness of the proposed framework. The results confirm that the Sim-to-Real transfer is successful, providing empirical evidence for the applicability of our method in real-world scenarios. This study shows that enhancing RL robustness through action-space perturbations is feasible and effective. More importantly, our findings contribute to the future development of autonomous intelligence, particularly in improving its resilience to uncertainties and dynamic environments.

Details

ICRA Conference 2024 Conference Paper

Multi-Task Learning of Active Fault-Tolerant Controller for Leg Failures in Quadruped robots

Taixian Hou
Jiaxin Tu
Xiaofei Gao
Zhiyan Dong
Peng Zhai
Lihua Zhang 0002

Electric quadruped robots used in outdoor exploration are susceptible to leg-related electrical or mechanical failures. Unexpected joint power loss and joint locking can immediately pose a falling threat. Typically, controllers lack the capability to actively sense the condition of their own joints and take proactive actions. Maintaining the original motion patterns could lead to disastrous consequences, as the controller may produce irrational output within a short period of time, further creating the risk of serious physical injuries. This paper presents a hierarchical fault-tolerant control scheme employing a multi-task training architecture capable of actively perceiving and overcoming two types of leg joint faults. The architecture simultaneously trains three joint task policies for health, power loss, and locking scenarios in parallel, introducing a symmetric reflection initialization technique to ensure rapid and stable gait skill transformations. Experiments demonstrate that the control scheme is robust in unexpected scenarios where a single leg experiences concurrent joint faults in two joints. Furthermore, the policy retains the robot’s planar mobility, enabling rough velocity tracking. Finally, zero-shot Sim2Real transfer is achieved on the real-world SOLO8 robot, countering both electrical and mechanical failures.

Details

AAMAS Conference 2024 Conference Paper

Successively Pruned Q-Learning: Using Self Q-function to Reduce the Overestimation

Zhaolin Xue
Lihua Zhang
Zhiyan Dong

It’s well-known that the Q-learning algorithm suffers the overestimation owing to using the maximum state-action value as an approximation of the maximum expected state-action value. Double Q-learning and other algorithms have been proposed as efficient solutions to alleviate the overestimation. However, these proposed methods intend to utilize multiple Q-functions to reduce the overestimation and ignore the information of single Q-function. In this paper, 1) we reinterpret the update process of Q-learning, build a more precise model compatible with previous model. 2) We propose a novel and simple method to control the maximum bias by employing the information of single Q-function. 3) Our method not only balances between the overestimation and the underestimation, but also attains the minimum bias under proper hyper-parameters. 4) Moreover, it can be naturally generalized to the discrete control domain and continuous control tasks. We reveal that our algorithms outperform Double DQN and other algorithms on some representative games and some classical off-policy actor-critic algorithms can also gain benefits from our method.

PDF

AAAI Conference 2022 Conference Paper

Robust Adversarial Reinforcement Learning with Dissipation Inequation Constraint

Peng Zhai
Jie Luo
Zhiyan Dong
Lihua Zhang
Shunli Wang
Dingkang Yang

Robust adversarial reinforcement learning is an effective method to train agents to manage uncertain disturbance and modeling errors in real environments. However, for systems that are sensitive to disturbances or those that are difficult to stabilize, it is easier to learn a powerful adversary than establish a stable control policy. An improper strong adversary can destabilize the system, introduce biases in the sampling process, make the learning process unstable, and even reduce the robustness of the policy. In this study, we consider the problem of ensuring system stability during training in the adversarial reinforcement learning architecture. The dissipative principle of robust H∞ control is extended to the Markov Decision Process, and robust stability constraints are obtained based on L2 gain performance in the reinforcement learning system. Thus, we propose a dissipation-inequationconstraint-based adversarial reinforcement learning architecture. This architecture ensures the stability of the system during training by imposing constraints on the normal and adversarial agents. Theoretically, this architecture can be applied to a large family of deep reinforcement learning algorithms. Results of experiments in MuJoCo and GymFc environments show that our architecture effectively improves the robustness of the controller against environmental changes and adapts to more powerful adversaries. Results of the flight experiments on a real quadcopter indicate that our method can directly deploy the policy trained in the simulation environment to the real environment, and our controller outperforms the PID controller based on hardware-in-the-loop. Both our theoretical and empirical results provide new and critical outlooks on the adversarial reinforcement learning architecture from a rigorous robust control perspective.

PDF Details