Author name cluster

Hui Cheng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

39 papers

2 author rows

IROS Conference 2025 Conference Paper

Biomechanically-Inspired Bipedal Robot Locomotion via Hybrid Gait Representation and Model-Guided Reinforcement Learning

Lijie Xie
Haomin Rong
Zujian Chen
Zida Zhou
Shaolin Mo
Hui Cheng

Achieving stable and natural locomotion in bipedal robots, comparable to that of humans and animals, remains a long-standing challenge in robotics. In this work, we propose a bio-inspired low-level control framework that streamlines the generation of naturalistic gait patterns while ensuring adaptability. Our approach begins with the design of a low-dimensional gait representation that captures key characteristics of human and animal locomotion. This representation is then integrated with the Linear Inverted Pendulum Model (LIPM) to form an abstract yet effective motion descriptor. Serving as a kinematic reference within a reinforcement learning (RL) framework, this descriptor enables the training of control policies that strike a balance between biomechanical realism and adaptability. Rather than strictly adhering to predefined gait trajectories, the learned policies dynamically adjust to optimize both stability and velocity tracking. As a result, our method enables bipedal robots to exhibit smooth, biomechanically realistic locomotion while enhancing stability and adaptability. We validate the proposed framework through real-world experiments on our bipedal robot, demonstrating its ability to achieve stable and efficient locomotion.

IROS Conference 2025 Conference Paper

Design of an Affordable, Fully-Actuated Biomimetic Hand for Dexterous Teleoperation Systems

Zhaoliang Wan
Zida Zhou
Zetong Bi
Zehui Yang
Hao Ding
Hui Cheng

This paper addresses the scarcity of affordable, fully-actuated five-fingered hands for dexterous teleoperation, which is crucial for collecting large-scale real-robot data within the "Learning from Demonstrations" paradigm. We introduce the prototype version of the RAPID Hand, the first low-cost, 20-degree-of-actuation (DoA) dexterous hand that integrates a novel anthropomorphic actuation and transmission scheme with an optimized motor layout and structural design to enhance dexterity. Specifically, the RAPID Hand features a universal phalangeal transmission scheme for the non-thumb fingers and an omnidirectional thumb actuation mechanism. Prioritizing affordability, the hand employs 3D-printed parts combined with custom gears for easier replacement and repair. We assess the RAPID Hand’s performance through quantitative metrics and qualitative testing in a dexterous teleoperation system, which is evaluated on three challenging tasks: multi-finger retrieval, ladle handling, and human-like piano playing. The results indicate that the RAPID Hand’s fully actuated 20-DoF design holds significant promise for dexterous teleoperation.

ICRA Conference 2025 Conference Paper

Effective Heterogeneous Point Cloud-Based Place Recognition and Relative Localization for Ground and Aerial Vehicles

Rui Mao
Hui Cheng

Place recognition and relative localization are crucial for realizing the potential of collaboration in ground and aerial robot teams. Many existing works focus only on ground robots and are not well-suited for heterogeneous robot systems in large-scale environments. In this paper, we propose a novel pipeline based on BEV density image, combined with an enhanced data structure, for place recognition in air-ground robotic collaboration systems. An efficient height alignment algorithm is proposed for relative localization. Extensive experiments on various types of public datasets validate the efficacy of our method compared to other SOTA works. We also show that our method is capable to detect inter- and intra-robot loop closures in a ground and aerial multi-session SLAM system.

IROS Conference 2025 Conference Paper

Enhancing the Flexibility of a Quadruped Robot with a 2-DOF Active Spine Using Nonlinear Model Predictive Control

Zeyi Yang
Zhiyong Xu
Haoming Rong
Shaolin Mo
Yuying Chen
Zujian Chen
Tao Wang
Hui Cheng

For quadrupeds, a flexible spine allows them to traverse space and make quick turns. From the perspective of mechanical design in quadruped robots, an active spine with 2 degrees of freedom (2-DOF) can achieve dynamic posture adjustment similar to biological organisms which allows for pitch and yaw control. In this work, we present a novel approach to enhance the flexibility of a quadruped robot, Yatsen Lion II, by incorporating a 2-DOF active spine, which is mechanically designed as a linkage-driven parallelogram mechanism. To optimize its motion, we utilize nonlinear model predictive control (NMPC), which combines centroidal dynamics with full kinematics. By incorporating the two extra DOFs of the spinal joint into the generalized coordinates and velocities, we represent the robot as a hybrid dynamic system, capturing the intricate interplay between the legs and spine. Centroidal dynamics act as a crucial bridge between joint movements and the robot’s overall momentum, enabling the controller to synchronize the quadruped’s movements with dynamic spinal adjustments and adaptive gait patterns. We validate our approach through both simulation and real-world experiments. We compare spinal quadruped robot to their rigid-spine counterparts across key locomotion metrics, including in-place turning, straight-line speed, and turning radius. The results indicate that the spined quadrupedal robot outperforms its rigid counterpart by up to 26%, highlighting its flexibility.

NeurIPS Conference 2025 Conference Paper

RAPID Hand: Robust, Affordable, Perception-Integrated, Dexterous Manipulation Platform for Embodied Intelligence

Zhaoliang Wan
Zetong Bi
Zida Zhou
Hao Ren
Yiming Zeng
Yihan Li
Lu Qi
Xu Yang

This paper addresses the scarcity of low-cost but high-dexterity platforms for collecting real-world multi-fingered robot manipulation data towards generalist robot autonomy. To achieve it, we propose the RAPID Hand, a co-optimized hardware and software platform where the compact 20-DoF hand, robust whole-hand perception, and high-DoF teleoperation interface are jointly designed. Specifically, RAPID Hand adopts a compact and practical hand ontology and a hardware-level perception framework that stably integrates wrist-mounted vision, fingertip tactile sensing, and proprioception with sub-7 ms latency and spatial alignment. Collecting high-quality demonstrations on high-DoF hands is challenging, as existing teleoperation methods struggle with precision and stability on complex multi-fingered systems. We address this by co-optimizing hand design, perception integration, and teleoperation interface through a universal actuation scheme, custom perception electronics, and two retargeting constraints. We evaluate the platform’s hardware, perception, and teleoperation interface. Training a diffusion policy on collected data shows superior performance over prior works, validating the system’s capability for reliable, high-quality data collection. The platform is constructed from low-cost and off-the-shelf components and will be made public to ensure reproducibility and ease of adoption.

ICRA Conference 2025 Conference Paper

RM-Planner: Integrating Reinforcement Learning with Whole-Body Model Predictive Control for Mobile Manipulation

Zixuan Zhuang
Le Zheng
Wanyue Li
Renming Liu
Peng Lu
Hui Cheng

Mobile manipulation is a crucial problem in various real-world applications. However, existing methods have demonstrated unsatisfactory training efficiency and sparse rewards, requiring complex coordination strategies between the mobile base and arm. In this paper, we propose RM-Planner, a planning method for mobile manipulation tasks in unknown complex environments. By adopting a two-layer hierarchical framework, we utilize a whole-body Model Predictive Control (MPC)-based low-level planner to track subgoals and generate aggressive but safe joint commands throughout the entire manipulation process, while a Reinforcement Learning (RL)based high-level policy directly uses 3D point cloud representations of the environment, guiding the robot to achieve optimal manipulation postures based on current observations and specific task objectives. We conduct extensive simulations and real-world experiments, where RM-planner significantly outperforms state-of-the-art methods. Our code will be released at https://github.com/SYSU-RoboticsLab/RM-Planner.git.

IROS Conference 2025 Conference Paper

RMCC: Rigid Multi-joint Coupled Continuum Structure for Bionic Robots

Zida Zhou
Ying Wu
Zujian Chen
Zetong Bi
Hui Cheng

Continuum robots, inspired by biological structures such as spines and tails, have attracted significant attention due to their flexibility and ability to perform complex tasks in confined and dynamic environments. However, traditional flexible continuum robots often encounter challenges such as non-linearity, hysteresis, and limited load-bearing capacity, which can compromise their precision and effectiveness in practical applications. To address these limitations, this paper presents a novel bionic continuum mechanism: Rigid Multi-joint Coupled Continuum Structure(RMCC), which employs a rigid mechanical transmission mode to couple all joints, achieving coordinated movement of multiple joints. Its rigid structural composition and transmission method provide it with high precision and load capacity. The coordinated motion of the joints endows it with the dexterity of a continuum mechanism, while also enabling efficient and precise control with a minimal number of motors. The modular joint design improves the system’s scalability and adaptability, enabling a wide range of configurations to suit diverse robotic applications. The feasibility and effectiveness of the proposed system are validated through a series of bio-inspired experiments, including lizardlike crawling, falling-cat movement, and adaptive grasping like birds. The experimental results confirm that the RMCC exhibits the flexibility and adaptability of animals, demonstrating its potential for diverse bionic robotics applications.

ICLR Conference 2025 Conference Paper

SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting

Huajian Huang
Yingshu Chen
Longwei Li
Hui Cheng
Tristan Braud
Yajie Zhao
Sai-Kit Yeung

360-degree cameras streamline data collection for radiance field 3D reconstruction by capturing comprehensive scene data. However, traditional radiance field methods do not address the specific challenges inherent to 360-degree images. We present SC-OmniGS, a novel self-calibrating omnidirectional Gaussian splatting system for fast and accurate omnidirectional radiance field reconstruction using 360-degree images. Rather than converting 360-degree images to cube maps and performing perspective image calibration, we treat 360-degree images as a whole sphere and derive a mathematical framework that enables direct omnidirectional camera pose calibration accompanied by 3D Gaussians optimization. Furthermore, we introduce a differentiable omnidirectional camera model in order to rectify the distortion of real-world data for performance enhancement. Overall, the omnidirectional camera intrinsic model, extrinsic poses, and 3D Gaussians are jointly optimized by minimizing weighted spherical photometric loss. Extensive experiments have demonstrated that our proposed SC-OmniGS is able to recover a high-quality radiance field from noisy camera poses or even no pose prior in challenging scenarios characterized by wide baselines and non-object-centric configurations. The noticeable performance gain in the real-world dataset captured by consumer-grade omnidirectional cameras verifies the effectiveness of our general omnidirectional camera model in reducing the distortion of 360-degree images.

IROS Conference 2025 Conference Paper

Seamless Transition Control in Spring-Legged Quadrotors: A Hybrid Dynamics Perspective with Guaranteed Feasibility

Hongli Li
Botao Zhang
Rui Mao
Tao Wang
Hui Cheng

Legged aerial-terrestrial robots have garnered significant research attention in recent years due to their enhanced environmental adaptability through combined aerial and terrestrial locomotion. However, existing passive spring-legged aerial robots exhibit limited motion versatility, demonstrating single stance gait during ground impacts, which constrains their task adaptability and creates substantial challenges in hybrid trajectory optimization and switching control. To address these difficulties, this work presents a systematic solution to achieve diverse hybrid locomotion. We innovatively establish the differential flatness property for spring-legged quadrotors in both aerial and terrestrial domains, and propose a unified hybrid trajectory optimization framework that generates smooth, agile, and dynamically feasible multi-modal trajectories incorporating diverse stance gait patterns. Furthermore, a hybrid nonlinear model predictive controller with a trajectory extension strategy is developed to enhance hybrid tracking precision and mode transition execution. Compared to existing methods, we achieve a 27% reduction in tracking error during hybrid locomotion while maintaining high-precision foot placement. The source code will be released to benefit the community 1

IROS Conference 2025 Conference Paper

TOPP-DWR: Time-Optimal Path Parameterization of Differential-Driven Wheeled Robots Considering Piecewise-Constant Angular Velocity Constraints

Yong Li
Yujun Huang
Yi Chen
Hui Cheng

Differential-driven wheeled robots (DWR) represent the quintessential type of mobile robots and find extensive applications across the robotic field. Most high-performance control approaches for DWR explicitly utilize the linear and angular velocities of the trajectory as control references. However, existing research on time-optimal path parameterization (TOPP) for mobile robots usually neglects the angular velocity and joint velocity constraints, which can result in degraded control performance in practical applications. In this article, a systematic and practical TOPP algorithm named TOPP-DWR is proposed for DWR and other mobile robots. First, the non-uniform B-spline is adopted to represent the initial trajectory in the task space. Second, the piecewise-constant angular velocity, as well as joint velocity, linear velocity, and linear acceleration constraints, are incorporated into the TOPP problem. During the construction of the optimization problem, the aforementioned constraints are uniformly represented as linear velocity constraints. To boost the numerical computational efficiency, we introduce a slack variable to reformulate the problem into second-order-cone programming (SOCP). Subsequently, comparative experiments are conducted to validate the superiority of the proposed method. Quantitative performance indexes show that TOPP-DWR achieves TOPP while adhering to all constraints. Finally, field autonomous navigation experiments are carried out to validate the practicability of TOPP-DWR in real-world applications.

AAAI Conference 2024 Conference Paper

FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels

Jichang Li
Guanbin Li
Hui Cheng
Zicheng Liao
Yizhou Yu

Federated Learning with Noisy Labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning by aggregating multiple client models trained with local noisy or clean samples. On the basis of a federated learning framework, recent advances primarily adopt label noise filtering to separate clean samples from noisy ones on each client, thereby mitigating the negative impact of label noise. However, these prior methods do not learn noise filters by exploiting knowledge across all clients, leading to sub-optimal and inferior noise filtering performance and thus damaging training stability. In this paper, we present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter for effectively identifying samples with noisy labels on every client, thereby raising stability during local training sessions. Without sacrificing data privacy, this is achieved by modeling the global distribution of label noise across all clients. Then, in an effort to make the global model achieve higher performance, we introduce a Predictive Consistency based Sampler to identify more credible local data for local model training, thus preventing noise memorization and further boosting the training stability. Extensive experiments on CIFAR-10, CIFAR-100, and Clothing1M demonstrate that FedDiv achieves superior performance over state-of-the-art F-LNL methods under different label noise settings for both IID and non-IID data partitions. Source code is publicly available at https://github.com/lijichang/FLNL-FedDiv.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation

Junhao Cai
Yuji Yang
Weihao Yuan
Yisheng He
Zilong Dong
Liefeng Bo
Hui Cheng
Qifeng Chen

This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework that leverages 3D Gaussian representation to not only capture explicit shapes but also enable the simulated continuum to render object masks as 2D shape surrogates during training. We propose a new dynamic 3D Gaussian framework based on motion factorization to recover the object as 3D Gaussian point sets across different time states. Furthermore, we develop a coarse-to-fine filling strategy to generate the density fields of the object from the Gaussian reconstruction, allowing for the extraction of object continuums along with their surfaces and the integration of Gaussian attributes into these continuum. In addition to the extracted object surfaces, the Gaussian-informed continuum also enables the rendering of object masks during simulations, serving as 2D-shape guidance for physical property estimation. Extensive experimental evaluations demonstrate that our pipeline achieves state-of-the-art performance across multiple benchmarks and metrics. Additionally, we illustrate the effectiveness of the proposed method through real-world demonstrations, showcasing its practical utility. Our project page is at https: //jukgei. github. io/project/gic.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Robust and Energy-Efficient Control for Multi-task Aerial Manipulation with Automatic Arm-switching

Ying Wu
Zida Zhou
Mingxin Wei
Hui Cheng

Aerial manipulation has received increasing research interest with wide applications of drones. To perform specific tasks, robotic arms with various mechanical structures will be mounted on the drone. It results in sudden disturbances to the aerial manipulator when switching the robotic arm or interacting with the environment. Hence, it is challenging to design a generic and robust control strategy adapted to various robotic arms when achieving multi-task aerial manipulation. In this paper, we present a learning-based control algorithm that allows online trajectory optimization and tracking to accomplish various aerial interaction tasks without manual adjustment. The proposed energy-saved trajectory planning approach integrates coupled dynamics model with a single rigid body to generate the energy-efficient trajectory for the aerial manipulator. Addressing the challenges of precise control when performing aerial manipulation tasks, this paper presents a controller based on deep neural networks that classifies and learns accurate forces and moments caused by different robotic arms and interactions. Moreover, the forces arising from robotic arm motions are delicately used as part of the drone’s power to save energy. Extensive real-world experiments demonstrate that the proposed method can adapt to various robotic arms and interactions when performing multi-task aerial manipulation.

ICRA Conference 2024 Conference Paper

Robust Control for Bidirectional Thrust Quadrotors under Instantaneously Drastic Disturbances

Zujian Chen
Shaolin Mo
Botao Zhang
Jiyu Li
Hui Cheng

Quadrotors may crash and cause severe accidents under instantaneously drastic disturbances. To mitigate the effect of such disturbances, these critical issues should be considered: efficient disturbance observation and compensation, full attitude controllability, and instant output power generation of the quadrotor. In this paper, to keep the quadrotor stable even under suddenly drastic disturbances, a novel control framework is presented to by integrating the advantages of active disturbance rejection control (ADRC) as well as geometric control for a quadrotor with bidirectional thrust capabilities. Moreover, to strengthen the adaptability under significant disturbances, a novel switching strategy is introduced into the control framework by virtue of the quadrotor’s bidirectional thrust capabilities. The ADRC scheme is performed when the disturbances are within a range; alternatively, if the disturbances surpass the preset range and the desired control is beyond the ultimate output of the quadrotor, the quadrotor compliantly responds by executing a 180° flip reverse flight to handle such drastic disturbances. Numerical and real-world experiments demonstrate that the proposed robust control strategy has superior performance adapts to instantaneously drastic disturbances.

IROS Conference 2024 Conference Paper

VRExplorer: An Efficient View-Region based Autonomous Exploration Method in Unknown Environments for UAV

Kai Xu
Lanxiang Zheng
Mingxin Wei
Hui Cheng

Autonomous exploration plays a crucial role in robotics applications like rescue and scene reconstruction. This work addresses the challenges of autonomous exploration in intricate unknown environments by presenting a novel UAV autonomous exploration method based on a new concept of the view-region. Our proposed approach leverages the view-region to replace the conventional viewpoint generation and selection process, streamlining the planning process for exploration. Simultaneously, we model the problem of maximizing frontier coverage within the field of view during exploration, and jointly optimize it with the exploration path optimization problem. This approach ensures exploration path safety and effectiveness while being aggressive. Additionally, a gimbal is incorporated beneath the camera, with an associated optimization problem designed to minimize UAV self-rotation and enhance exploration efficiency. Simulations and real-world experiments demonstrate that the proposed method outperforms existing state-of-the-art methods in terms of runtime and distance traveled.

ICRA Conference 2023 Conference Paper

Dynamic Locomotion of a Quadruped Robot with Active Spine via Model Predictive Control

Wanyue Li
Zida Zhou
Hui Cheng

As an active spine introduces more degree of freedoms (DOFs) as well as time-varying inertia, locomotion control of spined quadruped robots is challenging. Direct optimization on the full dynamics model causes prohibitive calculation time and is difficult to apply to embedded platforms. Model predictive control (MPC)-based on SRB dynamics is a prevalent approach for ordinary quadruped robots, regarding the whole robot as a single rigid body (SRB). However, the approach ignores the changes of the center of mass (CoM) and inertia, which seriously affects the robot's stability and could not be used in spined quadruped robots directly. To resolve the above issue, this paper presents an MPC approach that considers the movements of the spine in the SRB model. Since the mass of the robot is concentrated on its body, the whole robot is modelled as an unactuated SRB with fully-actuated internal spine joints. MPC finds the optimal ground reaction forces (GRFs) based on the SRB dynamics, in which the missing spine part is complemented by the pre-defined spine joints' states and corresponding inertia sequence. According to the GRFs, the full dynamic model calculates the precise joint torques. In addition, a quadruped robot with a 3-DOF active spine, Yat-sen Lion, is developed. With the presented approach, experimental results illustrate that Yat-sen Lion freely achieves bending, arching, and turning behaviors while trotting at speeds of 3. 8 m/s in simulations and 0. 5 m/s in real-world experiments.

ICRA Conference 2023 Conference Paper

Unidirectional-Road-Network-Based Global Path Planning for Cleaning Robots in Semi-Structured Environments

Yong Li
Hui Cheng

Practical global path planning is critical for commercializing cleaning robots working in semi-structured environments. In the literature, global path planning methods for free space usually focus on path length and neglect the traffic rule constraints of the environments, which leads to high-frequency re-planning and increases collision risks. In contrast, those for structured environments are developed mainly by strictly complying with the road network representing the traffic rule constraints, which may result in an overlong path that hinders the overall navigation efficiency. This article proposes a general and systematic approach to improve global path planning performance in semi-structured environments. A unidirectional road network is built to represent the traffic constraints in semi-structured environments and a hybrid strategy is proposed to achieve a guaranteed planning result. Cutting across the road at the starting and the goal points are allowed to achieve a shorter path. Especially, a two-layer potential map is proposed to achieve a guaranteed performance when the starting and the goal points are in complex intersections. Comparative experiments are carried out to validate the effectiveness of the proposed method. Quantitative experimental results show that, compared with the state-of-art, the proposed method guarantees a much better balance between path length and the consistency with the road network.

ICRA Conference 2022 Conference Paper

Decentralized Global Connectivity Maintenance for Multi-Robot Navigation: A Reinforcement Learning Approach

Minghao Li
Yingrui Jie
Yang Kong
Hui Cheng

The problem of multi-robot navigation of connectivity maintenance is challenging in multi-robot applications. This work investigates how to navigate a multi-robot team in unknown environments while maintaining connectivity. We propose a reinforcement learning (RL) approach to develop a decentralized policy, which is shared among multiple robots. Given range sensor measurements and the positions of other robots, the policy aims to generate control commands for navigation and preserve the global connectivity of the robot team. We incorporate connectivity concerns into the RL framework as constraints and introduce behavior cloning to reduce the exploration complexity of policy optimization. The policy is optimized with all transition data collected by multiple robots in random simulated scenarios. We validate the effectiveness of the proposed approach by comparing different combinations of connectivity constraints and behavior cloning. We also show that our policy can generalize to unseen scenarios in both simulation and holonomic robots experiments.

NeurIPS Conference 2022 Conference Paper

Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning

Ziyi Zhang
Weikai Chen
Hui Cheng
Zhen Li
Siyuan Li
Liang Lin
Guanbin Li

We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source pretrained model is adapted to the target domain without access to the source data. Existing techniques mainly leverage self-supervised pseudo-labeling to achieve class-wise global alignment [1] or rely on local structure extraction that encourages the feature consistency among neighborhoods [2]. While impressive progress has been made, both lines of methods have their own drawbacks – the “global” approach is sensitive to noisy labels while the “local” counterpart suffers from the source bias. In this paper, we present Divide and Contrast (DaC), a new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. Based on the prediction confidence of the source model, DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals under an adaptive contrastive learning framework. Specifically, the source-like samples are utilized for learning global class clustering thanks to their relatively clean labels. The more noisy target-specific data are harnessed at the instance level for learning the intrinsic local structures. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch. Extensive experiments on VisDA, Office-Home, and the more challenging DomainNet have verified the superior performance of DaC over current state-of-the-art approaches. The code is available at https: //github. com/ZyeZhang/DaC. git.

EAAI Journal 2022 Journal Article

Safe learning-based gradient-free model predictive control based on cross-entropy method

Lei Zheng
Rui Yang
Zhixuan Wu
Jiesen Pan
Hui Cheng

In this paper, a safe and learning-based control framework for model predictive control (MPC) is proposed to optimize nonlinear systems with a non-differentiable objective function under uncertain environmental disturbances. The control framework integrates a learning-based MPC with an auxiliary controller in a way of minimal intervention. The learning-based MPC augments the prior nominal model with incremental Gaussian Processes to learn the uncertain disturbances. The cross-entropy method (CEM) is utilized as the sampling-based optimizer for the MPC with a non-differentiable objective function. A minimal intervention controller is devised with a control Lyapunov function and a control barrier function to guide the sampling process and endow the system with high probabilistic safety. The proposed algorithm shows a safe and adaptive control performance on a simulated quadrotor in the tasks of trajectory tracking and obstacle avoidance under uncertain wind disturbances.

ICRA Conference 2022 Conference Paper

Uncertainty-based Exploring Strategy in Densely Cluttered Scenes for Vacuum Cup Grasping

Kimwa Tung
Jingcheng Su
Junhao Cai
Zhaoliang Wan
Hui Cheng

Grasping a wide range of novel objects in densely cluttered scenes is difficult due to irregular shapes of objects and the uncertainty in sensing. In this paper, a novel vacuum cup grasping method, based on uncertainty modeling of perception data and grasp geometric heuristics, is proposed to grasp unknown objects in densely cluttered scenes. The probabilistic signed distance function is proposed to both reconstruct the point cloud of a scene and explicitly model the uncertainty from depth images captured from a low-cost stereo camera. The quasi-static spring model is used to approximate seal formation between the suction cup and the reconstructed point cloud. A coarse-to-fine exploration procedure is proposed to refine the estimated point cloud, reduce uncertainties during the movement of the robot and redetermine the target grasp pose iteratively. Extensive experiments show that our proposed method achieves state-of-the-art performance on real-world grasping and outperforms existing methods by a large margin.

ICRA Conference 2021 Conference Paper

Control of an Aerial Manipulator Using a Quadrotor with a Replaceable Robotic Arm

Zizhen Ouyang
Ruidong Mei
Zisen Liu
Mingxin Wei
Zida Zhou
Hui Cheng

Control of an aerial manipulator is challenging due to the decentralized dynamics of the aerial vehicle and the robotic arm. It is generally complex to adjust the controller of the aerial manipulator when replacing a different robotic arm. This paper presents a flexible control scheme for a quadrotor-based aerial manipulator equipped with a replaceable robotic arm. To analyze the dynamic characteristics during grasping, the model of the aerial manipulator is decentralized including the models of a quadrotor and the centroid of an n-DOF robotic arm. The interaction effect of a moving robotic arm on the quadrotor is considered by analyzing the varying centroid of the robotic arm. Based on the modeling of the aerial manipulator, a control scheme integrating a linear model predictive control (LMPC) and a feedforward controller is presented to accurately control the motion of the aerial platform. The LMPC controls the aerial vehicle to follow the desired trajectory, and a feedforward controller keeps the aerial platform hovering stably during grasping. Practical experiments with two different robotic arms are performed. Experimental results show that the proposed modeling and control scheme provides a flexible and effective approach for an aerial manipulator with a replaceable robotic arm.

ICRA Conference 2020 Conference Paper

CCAN: Constraint Co-Attention Network for Instance Grasping

Junhao Cai
Xuefeng Tao
Hui Cheng
Zhanpeng Zhang

Instance grasping is a challenging robotic grasping task when a robot aims to grasp a specified target object in cluttered scenes. In this paper, we propose a novel end-to-end instance grasping method using only monocular workspace and query images, where the workspace image includes several objects and the query image only contains the target object. To effectively extract discriminative features and facilitate the training process, a learning-based method, referred to as Constraint Co-Attention Network (CCAN), is proposed which consists of a constraint co-attention module and a grasp affordance predictor. An effective co-attention module is presented to construct the features of a workspace image from the extracted features of the query image. By introducing soft constraints into the co-attention module, it highlights the target object's features while trivializes other objects' features in the workspace image. Using the features extracted from the co-attention module, the cascaded grasp affordance interpreter network only predicts the grasp configuration for the target object. The training of the CCAN is totally based on simulated self-supervision. Extensive qualitative and quantitative experiments show the effectiveness of our method both in simulated and real-world environments even for totally unseen objects.

ICRA Conference 2020 Conference Paper

Learning Affordance Space in Physical World for Vision-based Robotic Object Manipulation

Huadong Wu
Zhanpeng Zhang
Hui Cheng
Kai Yang
Jiaming Liu
Ziying Guo

What is a proper representation for objects in manipulation? What would human try to perceive when manipulating a new object in a new environment? In fact, instead of focusing on the texture and illumination, human can infer the "affordance" [36] of the objects from vision. Here "affordance" describes the object's intrinsic property that affords a particular type of manipulation. In this work, we investigate whether such affordance can be learned by a deep neural network. In particular, we propose an Affordance Space Perception Network (ASPN) that takes an image as input and outputs an affordance map. Different from existing works that infer the pixel-wise probability affordance map in image space, our affordance is defined in the real world space, thus eliminates the need of hand-eye calibration. In addition, we extend the representation ability of affordance by defining it in a 3D affordance space and propose a novel training strategy to improve the performance. Trained purely with simulation data, ASPN can achieve significant performance in the real world. It is a task-agnostic framework and can handle different objects, scenes and viewpoints. Extensive real-world experiments demonstrate the accuracy and robustness of our approach. We achieve the success rates of 94. 2% for singular-object pushing and 92. 4% for multiple-object pushing. We also achieve the success rates of 97. 2% for singular-object grasping and 95. 4% for multiple-object grasping, which outperform current state-of-the-art methods.

ICRA Conference 2019 Conference Paper

MetaGrasp: Data Efficient Grasping by Affordance Interpreter Network

Junhao Cai
Hui Cheng
Zhanpeng Zhang
Jingcheng Su

Data-driven approach for grasping shows significant advance recently. But these approaches usually require much training data. To increase the efficiency of grasping data collection, this paper presents a novel grasp training system including the whole pipeline from data collection to model inference. The system can collect effective grasp sample with a corrective strategy assisted by antipodal grasp rule, and we design an affordance interpreter network to predict pixelwise grasp affordance map. We define graspability, ungraspability and background as grasp affordances. The key advantage of our system is that the pixel-level affordance interpreter network trained with only a small number of grasp samples under antipodal rule can achieve significant performance on totally unseen objects and backgrounds. The training sample is only collected in simulation. Extensive qualitative and quantitative experiments demonstrate the accuracy and robustness of our proposed approach. In the real-world grasp experiments, we achieve a grasp success rate of 93% on a set of household items and 91% on a set of adversarial items with only about 6, 300 simulated samples. We also achieve 87% accuracy in clutter scenario. Although the model is trained using only RGB image, when changing the background textures, it also performs well and can achieve even 94% accuracy on the set of adversarial objects, which outperforms current state-of-the-art methods.

IROS Conference 2019 Conference Paper

PPR-Net: Point-wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-picking Scenarios

Zhi-Kai Dong
Sicheng Liu
Tao Zhou
Hui Cheng
Long Zeng 0001
Xingyao Yu
Houde Liu

Accurate object 6D pose estimation is a core task for robot bin-picking applications, especially when objects are randomly stacked with heavy occlusion. To address this problem, this paper proposes a simple but novel Point-wise Pose Regression Network (PPR-Net). For each point in the point cloud, the network regresses a 6D pose of the object instance that the point belongs to. We argue that the regressed poses of points from the same object instance should be located closely in pose space. Thus, these points can be clustered into different instances and their corresponding objects’ 6D poses can be estimated simultaneously. In our experiments, PPR-Net outperforms the state-of-the-art approach by 15% - 41% in average precision when evaluated on the benchmark Siléane dataset. In addition, it also works well in real world robot bin-picking tasks.

ICRA Conference 2018 Conference Paper

Avoidance of High-Speed Obstacles Based on Velocity Obstacles

Zhongchang Liu
Zeyu Jiang
Tianye Xu
Hui Cheng
Zhipeng Xie
Liang Lin

For obstacles moving with high speeds, existing motion planning methods can rarely guarantee collision avoidance. This paper proposes a viable two-period velocity obstacle algorithm where one period predicts potential collisions within a limited time horizon, and the second period foresees collisions beyond that horizon. The second period is activated only when the obstacle's moving speed is larger than the maximum speed of the robot. The applicability of the new algorithm and the related computation issues are discussed. Both computer simulations and laboratory experiments illustrated the effectiveness of the proposed obstacle avoidance algorithm.

IJCAI Conference 2018 Conference Paper

Deep Reasoning with Knowledge Graph for Social Relationship Understanding

Zhouxia Wang
Tianshui Chen
Jimmy Ren
Weihao Yu
Hui Cheng
Liang Lin

Social relationships (e. g. , friends, couple etc. ) form the basis of the social network in our daily life. Automatically interpreting such relationships bears a great potential for the intelligent systems to understand human behavior in depth and to better interact with people at a social level. Human beings interpret the social relationships within a group not only based on the people alone, and the interplay between such social relationships and the contextual information around the people also plays a significant role. However, these additional cues are largely overlooked by the previous studies. We found that the interplay between these two factors can be effectively modeled by a novel structured knowledge graph with proper message propagation and attention. And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects. Meanwhile, a graph attentional mechanism is introduced to explicitly reason about the discriminative objects to promote recognition. Extensive experiments on the public benchmarks demonstrate the superiority of our method over the existing leading competitors.

IROS Conference 2018 Conference Paper

Embedding Temporally Consistent Depth Recovery for Real-time Dense Mapping in Visual-inertial Odometry

Hui Cheng
Zhuoqi Zheng
Jinhao He
Chongyu Chen
Keze Wang
Liang Lin

Dense mapping is always the desire of simultaneous localization and mapping (SLAM), especially for the applications that require fast and dense scene information. Visual-inertial odometry (VIO) is a light-weight and effective solution to fast self-localization. However, VIO-based SLAM systems have difficulty in providing dense mapping results due to the spatial sparsity and temporal instability of the VIO depth estimations. Although there have been great efforts on real-time mapping and depth recovery from sparse measurements, the existing solutions for VIO-based SLAM still fail to preserve sufficient geometry details in their results. In this paper, we propose to embed depth recovery into VIO-based SLAM for real-time dense mapping. In the proposed method, we present a subspace-based stabilization scheme to maintain the temporal consistency and design a hierarchical pipeline for edge-preserving depth interpolation to reduce the computational burden. Numerous experiments demonstrate that our method can achieve an accuracy improvement of up to 49. 1 cm compared to state-of-the-art learning-based methods for depth recovery and reconstruct sufficient geometric details in dense mapping when only 0. 07% depth samples are available. Since a simple CPU implementation of our method already runs at 10-20 fps, we believe our method is very favorable for practical SLAM systems with critical computational requirements.

ICRA Conference 2018 Conference Paper

Fusing Object Context to Detect Functional Area for Cognitive Robots

Hui Cheng
Junhao Cai
Quande Liu
Zhanpeng Zhang
Kai Yang 0001
Chen Change Loy
Liang Lin

A cognitive robot usually needs to perform multiple tasks in practice and needs to locate the desired area for each task. Since deep learning has achieved substantial progress in image recognition, to solve this area detection problem, it is straightforward to label a functional area (affordance) image dataset and apply a well-trained deep-model-based classifier on all the potential image regions. However, annotating the functional area is time consuming and the requirement of large amount of training data limits the application scope. We observe that the functional area are usually related to the surrounding object context. In this work, we propose to use the existing object detection dataset and employ the object context as effective prior to improve the performance without additional annotated data. In particular, we formulate a two-stream network that fuses the object-related and functionality-related feature for functional area detection. The whole system is formulated in an end-to-end manner and easy to implement with current object detection framework. Experiments demonstrate that the proposed network outperforms current method by almost 20% in terms of precision and recall.

IROS Conference 2017 Conference Paper

An autonomous vision-based target tracking system for rotorcraft unmanned aerial vehicles

Hui Cheng
Lishan Lin
Zhuoqi Zheng
Yuwei Guan
Zhongchang Liu

In this paper, an autonomous vision-based tracking system is presented to track a maneuvering target for a rotorcraft unmanned aerial vehicle (UAV) with an onboard gimbal camera. To handle target occlusions or loss for real-time tracking, a robust and computationally efficient visual tracking scheme is considered using the Kernelized Correlation Filter (KCF) tracker and the redetection algorithm. The states of the target are estimated from the visual information. Moreover, feedback control laws of the gimbal and the UAV using the estimated states are proposed for the UAV to track the moving target autonomously. The algorithms are implemented on an onboard TK1 computer, and extensive outdoor flight experiments have been performed. Experimental results show that the proposed computationally efficient visual tracking scenario can stably track a maneuvering target and is robust to target occlusions and loss.

IROS Conference 2017 Conference Paper

Decentralized navigation of multiple agents based on ORCA and model predictive control

Hui Cheng
Qiyuan Zhu
Zhongchang Liu
Tianye Xu
Liang Lin

This paper presents a decentralized strategy for collision-free navigation of multiple agents. This strategy combines the Optimal Reciprocal Collision Avoidance (ORCA) algorithm and Model Predictive Control (MPC). Concretely, each agent applies the decentralized ORCA algorithm to compute the collision-avoiding velocities with respect to its neighbors. The derived velocities serve as constraints of a MPC problem whose solution provides the optimal control input that can ensure optimal motion of the agent. The states predicted from the agents' dynamic models are used in the ORCA algorithm to compute the ORCA velocity regions in future steps. This ORCA-MPC combined approach doesn't need a priori the preferred velocity of each agent in comparison to the traditional ORCA algorithm and its existing variants. Simulation results illustrate the effectiveness of the proposed method, and show that this new algorithm can reduce velocity vibrations in the traditional ORCA algorithm.

AAAI Conference 2016 Conference Paper

Zero-Shot Event Detection by Multimodal Distributional Semantic Embedding of Videos

Mohamed Elhoseiny
Jingen Liu
Hui Cheng
Harpreet Sawhney
Ahmed Elgammal

We propose a new zero-shot Event Detection method by Multi-modal Distributional Semantic embedding of videos. Our model embeds object and action concepts as well as other available modalities from videos into a distributional semantic space. To our knowledge, this is the ﬁrst Zero-Shot event detection model that is built on top of distributional semantics and extends it in the following directions: (a) semantic embedding of multimodal information in videos (with focus on the visual modalities), (b) automatically determining relevance of concepts/attributes to a free text query, which could be useful for other applications, and (c) retrieving videos by free text event query (e. g. , ”changing a vehicle tire”) based on their content. We embed videos into a distributional semantic space and then measure the similarity between videos and the event query in a free text form. We validated our method on the large TRECVID MED (Multimedia Event Detection) challenge. Using only the event title as a query, our method outperformed the state-of-the-art that uses big descriptions from 12. 6% to 13. 5% with MAP metric and 0. 73 to 0. 83 with ROC-AUC metric. It is also an order of magnitude faster.

EAAI Journal 2013 Journal Article

Multi-robot navigation based QoS routing in self-organizing networks

Xingwei Wang
Hui Cheng
Min Huang

The technical development drives the future networks to become large-scale, heterogeneous, and dynamic. Bio-inspired networking can help reduce the time-space complexity of the complex network. Due to the good features such as self-organization and self-management, self-organizing network (SON) will most probably be a priority choice for the next generation network. In this paper, a swarm intelligence based Quality of Service (QoS) routing protocol is proposed for SON. The inaccurate routing and QoS information is described with fuzzy mathematics whilst the utilities of both the user and the network service provider are considered by applying game theory. Based on the multi-robot navigation algorithm, the protocol is able to search a routing path which can satisfy the user QoS requirements and achieve the Pareto optimal utilities of the user and the network service provider under Nash equilibrium. The proposed protocol is implemented and evaluated by extensive simulation experiments. The results show that it beats both other swarm intelligence based routing protocols and the traditional Dijkstra algorithm based routing protocol. The searched routing paths support the win–win effect for both the user and the network service provider.

EAAI Journal 2010 Journal Article

Genetic algorithms with immigrants schemes for dynamic multicast problems in mobile ad hoc networks

Hui Cheng
Shengxiang Yang

In this paper, the problem of dynamic quality-of-service (QoS) multicast routing in mobile ad hoc networks is investigated. Lots of interesting works have been done on multicast since it is proved to be a NP-hard problem. However, most of them consider the static network scenarios only and the multicast tree cannot adapt to the topological changes. With the advancement in communication technologies, more and more wireless mobile networks appear, e. g. , mobile ad hoc networks (MANETs). In a MANET, the network topology keeps changing due to its inherent characteristics such as the node mobility and energy conservation. Therefore, an effective multicast algorithm should track the topological changes and adapt the best multicast tree to the changes accordingly. In this paper, we propose to use genetic algorithms with immigrants schemes to solve the dynamic QoS multicast problem in MANETs. MANETs are considered as target systems because they represent a new generation of wireless networks. In the construction of the dynamic network environments, two models are proposed and investigated. One is named as the general dynamics model in which the topologies are changed due to that the nodes are scheduled to sleep or wake up. The other is named as the worst dynamics model, in which the topologies are altered because some links on the current best multicast tree are removed. Extensive experiments are conducted based on both of the dynamic network models. The experimental results show that these immigrants based genetic algorithms can quickly adapt to the environmental changes (i. e. , the network topology changes) and produce high quality solutions following each change.

IROS Conference 2001 Conference Paper

Advantages and dynamics of parallel manipulators with redundant actuation

Hui Cheng
Guanfeng Liu 0002
Yiu Kuen Yiu
Zhenhua Xiong 0003
Zexiang Li 0001

Parallel manipulators with redundant actuation present both problems and advantages. We analyze some major advantages of redundancy to demonstrate its usefulness. The dynamics of closed-chain mechanisms has been studied by many researchers. However, they often only consider the case of normal actuation. Few methods are suitable for closed-chain mechanisms with redundant actuation. In the article, the dynamic model of redundantly actuated closed-chain mechanisms is presented explicitly by applying Lagrange-D'Alembert formulations. As an illustrative example, the dynamic model of a 2-DOF planar parallel redundant manipulator is presented. A PID control algorithm in the joint space is experimentally implemented on this parallel manipulator to perform trajectory tracking. The experimental results show the validity of PID control.

IROS Conference 2001 Conference Paper

Distribution of singularity and optimal control of redundant parallel manipulators

Guanfeng Liu 0002
Hui Cheng
Zhenhua Xiong 0003
X. Z. Wu
Y. L. Wu
Zexiang Li 0001

Singularity is a fundamental problem in the analysis of parallel mechanisms. The distribution of singularity in the workspace will determine to a great extent the properties of parallel mechanisms. We study the distribution of actuator singularity, which can also be applied to analyze end-effector singularity. A very important observation has been made that these two kinds of singularities are caused by the parameterization of a configuration manifold by actuator coordinates or end-effector coordinates. Despite the various styles of singularities of parallel mechanisms, there are some rules which govern the behavior of stable singularities. These rules provide some useful ideas in the design of redundant parallel mechanisms so as to achieve better performance in high speed motion and improve their stiffness. Optimal kinematic and dynamic control algorithms are designed and implemented which make use of the redundancy of the parallel mechanism. Experimental results agree with our expectation.

ICRA Conference 2001 Conference Paper

On the Dynamics of Parallel Manipulators

Yiu Kuen Yiu
Hui Cheng
Zhenhua Xiong 0003
Zexiang Li 0001
Guanfeng Liu 0002

Studies the dynamics of parallel manipulators. We first have a brief review and discussion on different dynamics formulations in the literature (Newton-Euler, direct Lagrangian, and Lagrange-D'Alembert formulation on the reduced system). Then we show the equivalence of these methods. Based on the concepts from differential manifolds, we prove that away from configuration singularity, there exists a projection from the joint space to parameterize the configuration space. The fact that the dynamics is well defined even at actuators singularity, end-effector singularity and other kinds of parameterization singularity is highlighted. For the method of reduced systems, there are two main drawbacks. Firstly the joints being cut for forming the tree system are presumed to have no external torque. Secondly the force and torque applied to other links of the manipulator is not considered. We propose two methods to remedy the situation. Firstly by cutting a link instead of a joint, all the joints torque can be incorporated into our equations of motion. This is useful not only for the case of actuating all the joints, but also if we consider compensating the joints friction. Secondly we propose a concept of transforming force to the generalized force space so that all the other forces and torque can be considered.

IROS Conference 1993 Conference Paper

A novel approach for motion planning

Hui Cheng
Heng-Da Cheng

The feasible map approach (FMA) to certain motion planning problems in robotics is introduced. This approach is based on the use of a feasible map representation of a configuration space. When a feasible map is constructed, the topology of a configuration space is derived by shrink transformation. Path planning is done at two levels: first, to find a feasible path in a feasible map, and second, to transform the feasible path in the feasible map into the configuration space. When a feasible map is obtained, the feasible path can be obtained according to any starting and desired configurations, at once. A detailed FMA is given on a sphere world and the performance of the proposed algorithm is demonstrated by experiments. The results indicate that the FMA is efficient, useful and widely applicable.