Author name cluster

Wei Zhang 0021

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

1 author row

IROS Conference 2025 Conference Paper

ALVO: Adaptive Learning with Velocity Obstacles for UGV Navigation in Dynamic Scenes

Yinduo Xie
Yuenan Zhao
Ran Song 0001
Zhiheng Li 0005
Lei Han 0001
Wei Zhang 0021

Autonomous navigation of unmanned ground vehicles (UGVs) in dynamic scenes is a challenging task that requires them to avoid obstacles and move toward the goal simultaneously. This paper proposes ALVO, an adaptive learning policy that leverages velocity obstacles for UGV navigation. ALVO employs an adaptive gating-based mechanism for reactive obstacle avoidance, which enables the UGV to either slow down or proactively navigate around obstacles based on the relative importance of the environmental state and the goal. A reward function based on velocity obstacles is also designed to guide the UGV to navigate toward the goal while avoiding obstacles. Extensive experiments demonstrate that ALVO outperforms the competing approaches in various dynamic environments. We also implemented our method on a real UGV and showed that it performed well in real-world scenarios.

Details

IROS Conference 2025 Conference Paper

VCADNet: Vision-based Circular Accessible Depth Prediction for UGV Perception

Tao Zhang
Yuenan Zhao
Xiaoyu Xu
Ran Song 0001
Lei Han 0001
Wei Zhang 0021

Circular accessible depth (CAD) provides a lightweight and robust traversability representation for autonomous navigation of unmanned ground vehicles (UGV). Aiming at the limitations of existing LiDAR-based methods in detecting low-thickness targets and executing semantic reasoning, we propose VCADNet, a vision-based neural network for circular accessible depth prediction. VCADNet comprises three core components: a geometry-based query module for multi-view bird’s eye view feature extraction, a polar coordinate transformation for CAD alignment, and a multi-scale U-Net architecture for depth prediction. In addition, we present a cross-modal contrastive learning scheme to enhance the spatial reasoning of VCADNet, which transfers knowledge from LiDAR-based encoders to vision-based counterparts. Extensive experiments demonstrate the superior performance of VCADNet in various UGV perception tasks.

Details

IROS Conference 2024 Conference Paper

Coarse-to-Fine Detection of Multiple Seams for Robotic Welding

Pengkun Wei
Shuo Cheng
Dayou Li
Ran Song 0001
Yipeng Zhang
Wei Zhang 0021

Efficiently detecting target weld seams while ensuring sub-millimeter accuracy has always been an important challenge in autonomous welding, which has significant application in industrial practice. Previous works mostly focused on recognizing and localizing welding seams one by one, leading to inferior efficiency in modeling the workpiece. This paper proposes a novel framework capable of multiple weld seams extraction using both RGB images and 3D point clouds. The RGB image is used to obtain the region of interest by approximately localizing the weld seams, and the point cloud is used to achieve the fine-edge extraction of the weld seams within the region of interest using region growth. Our method is further accelerated by using a pre-trained deep learning model to ensure both efficiency and generalization ability. The proposed method was comprehensively tested on various workpieces featuring both linear and curved weld seams, as well as in physical experiment systems. The results showcase considerable potential for real-world industrial applications, emphasizing the method’s efficiency and effectiveness. Videos of the real-world experiments can be found at https://youtu.be/pq162HSP2D4.

Details

ICML Conference 2024 Conference Paper

ESNet: Evolution and Succession Network for High-Resolution Salient Object Detection

Hongyu Liu 0003
Runmin Cong
Hua Li 0012
Qianqian Xu 0001
Qingming Huang
Wei Zhang 0021

Preserving details and avoiding high computational costs are the two main challenges for the High-Resolution Salient Object Detection (HRSOD) task. In this paper, we propose a two-stage HRSOD model from the perspective of evolution and succession, including an evolution stage with Low-resolution Location Model (LrLM) and a succession stage with High-resolution Refinement Model (HrRM). The evolution stage achieves detail-preserving salient objects localization on the low-resolution image through the evolution mechanisms on supervision and feature; the succession stage utilizes the shallow high-resolution features to complement and enhance the features inherited from the first stage in a lightweight manner and generate the final high-resolution saliency prediction. Besides, a new metric named Boundary-Detail-aware Mean Absolute Error (${MAE}_{{BD}}$) is designed to evaluate the ability to detect details in high-resolution scenes. Extensive experiments on five datasets demonstrate that our network achieves superior performance at real-time speed (49 FPS) compared to state-of-the-art methods.

Details

IROS Conference 2024 Conference Paper

MPGNet: Learning Move-Push-Grasping Synergy for Target-Oriented Grasping in Occluded Scenes

Dayou Li
Chenkun Zhao
Shuo Yang
Ran Song 0001
Xiaolei Li 0003
Wei Zhang 0021

This paper focuses on target-oriented grasping in occluded scenes, where the target object is specified by a binary mask and the goal is to grasp the target object with as few robotic manipulations as possible. Most existing methods rely on a push-grasping synergy to complete this task. To deliver a more powerful target-oriented grasping pipeline, we present MPGNet, a three-branch network for learning a synergy between moving, pushing, and grasping actions. We also propose a multi-stage training strategy to train the MPGNet which contains three policy networks corresponding to the three actions. The effectiveness of our method is demonstrated via both simulated and real-world experiments. Video of the real-world experiments is at https://youtu.be/S_QKZqkh0w8.

Details

IROS Conference 2024 Conference Paper

MPP: Multiscale Path Planning for UGV Navigation in Semi-structured Environments

Rui Cao
Zhiqiang Yang
Ran Song 0001
Ziyu Meng
Ruifeng Wang
Wei Zhang 0021

Autonomous navigation of unmanned ground vehicles (UGVs) in structured road and indoor environments has made significant progress in recent years. However, navigation in outdoor semi-structured environments remains a challenge. This paper presents the multiscale path planning (MPP) method for UGV navigation in semi-structured environments. MPP leverages global, mid-layer and local planners to obtain global path and handle local obstacles of different sizes. First, the global planner provides guidance based on road connection relationships, selecting optimal connections by evaluating the distance between road nodes. Next, the mid-layer planner perceives large-scale obstacles and constructs the costmap, generating a mid-layer path that offers a general direction for the UGV. Finally, a local trajectory planning algorithm, namely terrain-considering timed elastic band (TC-TEB), is used to obtain local trajectory. This algorithm incorporates terrain-velocity constraints into the TEB algorithm to ensure the vehicle’s vertical stability. We demonstrate the safety and effectiveness of MPP through experiments in both simulated and real-world environments.

Details

IROS Conference 2021 Conference Paper

A Hierarchical Framework for Quadruped Locomotion Based on Reinforcement Learning

Wenhao Tan
Xing Fang
Wei Zhang 0021
Ran Song 0001
Teng Chen
Yu Zheng 0001
Yibin Li 0001

Quadruped locomotion is a challenging task for learning-based algorithms. It requires tedious manual tuning and is difficult to deploy in reality due to the reality gap. In this paper, we propose a quadruped robot learning system for agile locomotion which does not require any pre-training and works well in various real-world terrains. We introduce a hierarchical learning framework that uses reinforcement learning as the high-level policy to adjust the low-level trajectory generator for better adaptability to the terrain. We compact the observation and action space of the reinforcement learning to deploy it on a host computer in reality. Besides, we design a trajectory generator guided by robot posture, which can generate adaptive foot trajectory to interact with the environment. Experimental results show that our system can be easily deployed in reality while only trained in simulation, and also has the advantages of fast convergence and good terrain adaptability. The supplementary video demonstration is available at https://vsislab.github.io/hfql/.

Details

ICRA Conference 2021 Conference Paper

Autonomous Multi-View Navigation via Deep Reinforcement Learning

Xueqin Huang
Wei Chen
Wei Zhang 0021
Ran Song 0001
Jiyu Cheng
Yibin Li 0001

In this paper, we propose a novel deep reinforcement learning (DRL) system for the autonomous navigation of mobile robots that consists of three modules: map navigation, multi-view perception and multi-branch control. Our DRL system takes as the input a routed map provided by a global planner and three RGB images captured by a multi-camera setup to gather global and local information, respectively. In particular, we present a multi-view perception module based on an attention mechanism to filter out redundant information caused by multi-camera sensing. We also replace raw RGB images with low-dimensional representations via a specifically designed network, which benefits a more robust sim2real transfer learning. Extensive experiments in both simulated and real-world scenarios demonstrate that our system outperforms state-of-the-art approaches.

Details

IROS Conference 2021 Conference Paper

PackerBot: Variable-Sized Product Packing with Heuristic Deep Reinforcement Learning

Zifei Yang
Shuo Yang
Shuai Song
Wei Zhang 0021
Ran Song 0001
Jiyu Cheng
Yibin Li 0001

Product packing is a typical application in ware-house automation that aims to pick objects from unstructured piles and place them into bins with optimized placing policy. However, it still remains a significant challenge to finish the product packing tasks in general logistics scenarios where the objects are variable-sized and the configurations are complex. In this work, we present the PackerBot, a complete robotic pipeline for performing variable-sized product packing in unstructured scenes. First, by leveraging the imperfect experience of human packer, we propose a heuristic DRL framework for learning optimal online 3D bin packing policy. Then we integrate it with a 6-DoF suction-based picking module and a product size estimation module, leading to a complete product packing system, namely the PackerBot. Extensive experimental results show that our method achieves the state-of-the-art performance in both simulated and real-world tests. The video demonstration is available at: https://vsislab.github.io/packerbot.

Details

IROS Conference 2020 Conference Paper

Autonomous Robot Navigation Based on Multi-Camera Perception

Kunyan Zhu
Wei Chen
Wei Zhang 0021
Ran Song 0001
Yibin Li 0001

In this paper, we propose an autonomous method for robot navigation based on a multi-camera setup that takes advantage of a wide field of view. A new multi-task network is designed for handling the visual information supplied by the left, central and right cameras to find the passable area, detect the intersection and infer the steering. Based on the outputs of the network, three navigation indicators are generated and then combined with the high-level control commands extracted by the proposed MapNet, which are finally fed into the driving controller. The indicators are also used through the controller for adjusting the driving velocity, which assists the robot to adjust the speed for smoothly bypassing obstacles. Experiments in real-world environments demonstrate that our method performs well in both local obstacle avoidance and global goal-directed navigation tasks.

Details

ICRA Conference 2020 Conference Paper

Cross-context Visual Imitation Learning from Demonstrations

Shuo Yang
Wei Zhang 0021
Weizhi Lu
Hesheng Wang 0001
Yibin Li 0001

Imitation learning enables robots to learn a task by simply watching the demonstration of the task. Current imitation learning methods usually require the learner and demonstrator to occur in the same context. This limits their scalability to practical applications. In this paper, we propose a more general imitation learning method which allows the learner and the demonstrator to come from different contexts, such as different viewpoints, backgrounds, and object positions and appearances. Specifically, we design a robotic system consisting of three models: context translation model, depth prediction model and multi-modal inverse dynamics model. First, the context translation model translates the demonstration to the context of learner from a different context. Then combining the color observation and depth observation as inputs, the inverse model maps the multi-modal observations into actions to reproduce the demonstration, where the depth observation is provided by a depth prediction model. By performing the block stacking tasks both in simulation and real world, we prove the cross-context learning advantage of the proposed robotic system over other systems.

Details

ICRA Conference 2020 Conference Paper

Grasp for Stacking via Deep Reinforcement Learning

Junhao Zhang
Wei Zhang 0021
Ran Song 0001
Lin Ma 0002
Yibin Li 0001

Integrated robotic arm system should contain both grasp and place actions. However, most grasping methods focus more on how to grasp objects, while ignoring the placement of the grasped objects, which limits their applications in various industrial environments. In this research, we propose a model-free deep Q-learning method to learn the grasping-stacking strategy end-to-end from scratch. Our method maps the images to the actions of the robotic arm through two deep networks: the grasping network (GNet) using the observation of the desk and the pile to infer the gripper’s position and orientation for grasping, and the stacking network (SNet) using the observation of the platform to infer the optimal location when placing the grasped object. To make a long-range planning, the two observations are integrated in the grasping for stacking network (GSN). We evaluate the proposed GSN on a grasping-stacking task in both simulated and real-world scenarios.

Details

IROS Conference 2020 Conference Paper

Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator

Yue Fan
Shilei Chu
Wei Zhang 0021
Ran Song 0001
Yibin Li 0001

We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less manual effort and high accuracy. Then a newly designed neural network is trained based on the annotated data to predict appropriate directions and translations for the drone to patrol in a lane-keeping manner as humans. Our method allows the drone to fly at a high altitude with a broad view and low risk. It can also detect all accessible directions at crossroads and further carry out the integration of available user instructions and autonomous patrolling control commands. Extensive experiments are conducted to demonstrate the accuracy of the proposed imitating learning process as well as the reliability of the holistic system for autonomous drone navigation. The codes, datasets as well as video demonstrations are available at https://vsislab.github.io/uavpatrol.

Details

IROS Conference 2019 Conference Paper

Learning Actions from Human Demonstration Video for Robotic Manipulation

Shuo Yang
Wei Zhang 0021
Weizhi Lu
Hesheng Wang 0001
Yibin Li 0001

Learning actions from human demonstration is an emerging trend for designing intelligent robotic systems, which can be referred as video to command. The performance of such approach highly relies on the quality of video captioning. However, the general video captioning methods focus more on the understanding of the full frame, lacking of consideration on the specific object of interests in robotic manipulations. We propose a novel deep model to learn actions from human demonstration video for robotic manipulation. It consists of two deep networks, grasp detection network (GNet) and video captioning network (CNet). GNet performs two functions: providing grasp solutions and extracting the local features for the object of interests in robotic manipulation. CNet outputs the captioning results by fusing the features of both full frames and local objects. Experimental results on UR5 robotic arm show that our method could produce more accurate command from video demonstration than state-of-the-art work, thereby leading to more robust grasping performance.

Details