Author name cluster

Zhichao Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

IROS Conference 2025 Conference Paper

6-DoF Object Tracking with Event-based Optical Flow and Frames

Zhichao Li
Arren Glover
Chiara Bartolozzi
Lorenzo Natale

Tracking the position and orientation of objects in space (i. e. , in 6-DoF) in real time is a fundamental problem in robotics for environment interaction. It becomes more challenging when objects move at high-speed due to frame rate limitations in conventional cameras and motion blur. Event cameras are characterized by high temporal resolution, low latency and high dynamic range, that can potentially overcome the impacts of motion blur. Traditional RGB cameras provide rich visual information that is more suitable for the challenging task of single-shot object pose estimation. In this work, we propose using event-based optical flow combined with an RGB based global object pose estimator for 6-DoF pose tracking of objects at high-speed, exploiting the core advantages of both types of vision sensors. Specifically, we propose an event-based optical flow algorithm for object motion measurement to implement an object 6-DoF velocity tracker. By integrating the tracked object 6-DoF velocity with low frequency estimated pose from the global pose estimator, the method can track pose when objects move at high-speed. The proposed algorithm is tested and validated on both synthetic and real world data, demonstrating its effectiveness, especially in high-speed motion scenarios.

Details

NeurIPS Conference 2025 Conference Paper

Object Concepts Emerge from Motion

Haoqian Liang
Xiaohui Wang
Zhichao Li
Ya Yang
Naiyan Wang

Object concepts play a foundational role in human visual cognition, enabling perception, memory, and interaction in the physical world. Inspired by findings in developmental psychology—where infants are shown to acquire object understanding through observation of motion—we propose a biologically inspired framework for learning object-centric visual representations in an unsupervised manner. We were inspired by the insight that motion boundary serves as a strong signal for object-level grouping, which can be used to derive pseudo-instance supervision from raw videos. Concretely, we generate motion-based instance masks using off-the-shelf optical flow and clustering algorithms, and use them to train visual encoders via contrastive learning. Our framework is fully label-free and does not rely on camera calibration, making it scalable to large-scale unstructured video data. We evaluate our approach on three downstream tasks spanning both low-level (monocular depth estimation) and high-level (3D object detection and occupancy prediction) vision. Our models outperform previous supervised and self-supervised baselines and demonstrate strong generalization to unseen scenes. These results suggest that motion-induced object representations offer a compelling alternative to existing vision foundation models, capturing a crucial but overlooked level of abstraction: the visual instance. The implementation can be found here: https: //github. com/yulemao/Object Concepts Emerge from Motion

PDF Details

ICRA Conference 2024 Conference Paper

EDOPT: Event-camera 6-DoF Dynamic Object Pose Tracking

Arren Glover
Luna Gava
Zhichao Li
Chiara Bartolozzi

High-frequency, low-latency, 6-DoF object tracking is useful for grasping objects in motion, taking robots beyond pick-and-place tasks. We propose using an event-camera for tracking the objects to leverage the low-latency and continuous (i. e. not fixed-rate) data capture for high-frequency tracking. We propose the EDOPT algorithm, which maintains real-time operation with a variable event-rate (which occurs due to variation in camera velocity and scene texture) and avoids frame-jumps and motion-blur which are problematic in traditional computer vision solutions. EDOPT uses a strong object prior, leading to a novel solution possible only with the event-camera. To our knowledge, this is the first method for 6-DoF object pose estimation with only the event-camera. The proposed method achieves comparable results to a state-of-the-art DNN technique that fuses frames, depth, and events. We demonstrate smooth, online object pose tracking with a live camera feed at > 300 Hz.

Details

ICRA Conference 2024 Conference Paper

Lightning NeRF: Efficient Hybrid Scene Representation for Autonomous Driving

Junyi Cao
Zhichao Li
Naiyan Wang
Chao Ma 0001

Recent studies have highlighted the promising application of NeRF in autonomous driving contexts. However, the complexity of outdoor environments, combined with the restricted viewpoints in driving scenarios, complicates the task of precisely reconstructing scene geometry. Such challenges often lead to diminished quality in reconstructions and extended durations for both training and rendering. To tackle these challenges, we present Lightning NeRF. It uses an efficient hybrid scene representation that effectively utilizes the geometry prior from LiDAR in autonomous driving scenarios. Lightning NeRF significantly improves the novel view synthesis performance of NeRF and reduces computational overheads. Through evaluations on real-world datasets, such as KITTI-360, Argoverse2, and our private dataset, we demonstrate that our approach not only exceeds the current state-of-the-art in novel view synthesis quality but also achieves a five-fold increase in training speed and a ten-fold improvement in rendering speed. Codes are available at https://github.com/VISION-SJTU/Lightning-NeRF.

Details

IROS Conference 2023 Conference Paper

Hybrid Object Tracking with Events and Frames

Zhichao Li
Nicola A. Piga
Franco Di Pietro
Massimiliano Iacono
Arren Glover
Lorenzo Natale
Chiara Bartolozzi

Robust object pose tracking plays an important role in robot manipulation, but it is still an open issue for quickly moving targets as motion blur and low frequency detection can reduce pose estimation accuracy even for state-of-the-art RGB-D-based methods. An event-camera is a low-latency vision sensor that can act complementary to RGB-D. Specifically, its sub-millisecond temporal resolution can be exploited to correct for pose estimation inaccuracies due to low frequency RGB-D based detection. To do so, we propose a dual Kalman filter: the first filter estimates an object's velocity from the spatiotemporal patterns of “events”, the second filter fuses the tracked object velocity with a low-frequency object pose estimated from a deep neural network using RGB-D data. The full system outputs high frequency, accurate object poses also for fast moving objects. The proposed method works towards low-power robotics by replacing high-cost GPU-based optical flow used in prior work with event-cameras that inherently extract the required signal without costly processing. The proposed algorithm achieves comparable or better performance when compared to two state-of-the-art 6-DoF object pose estimation algorithms and one hybrid event/RGB-D algorithm on benchmarks with simulated and real data. We discuss the benefits and tradeoffs for using the event-camera and contribute algorithm, code, and datasets to the community. The code and datasets are available at https://github.com/event-driven-robotics/Hybrid-object-tracking-with-events-and-frames.

Details

IROS Conference 2021 Conference Paper

Model-free Vehicle Tracking and State Estimation in Point Cloud Sequences

Ziqi Pang
Zhichao Li
Naiyan Wang

Estimating the states of surrounding traffic participants stays at the core of autonomous driving. In this paper, we study a novel setting of this problem: model-free single-object tracking (SOT), which takes the object state in the first frame as input, and jointly solves state estimation and tracking in subsequent frames. The main purpose for this new setting is to break the strong limitation of the popular "detection and tracking" scheme in multi-object tracking. Moreover, we notice that shape completion by overlaying the point clouds, which is a by-product of our proposed task, not only improves the performance of state estimation but also has numerous applications. As no benchmark for this task is available so far, we construct a new dataset LiDAR-SOT and corresponding evaluation protocols based on the Waymo Open dataset [29]. We then propose an optimization-based algorithm called SOTracker involving point cloud registration, vehicle shapes, correspondence, and motion priors. Our quantitative and qualitative results prove the effectiveness of our SOTracker and reveal the challenging cases for SOT in point clouds, including the sparsity of LiDAR data, abrupt motion variation, etc. Finally, we also explore how the proposed task and algorithm may benefit other autonomous driving applications, including simulating LiDAR scans, generating motion data, and annotating optical flow. The code and protocols for our benchmark and algorithm are available at https://github.com/TuSimple/LiDAR_SOT/.A video demonstration is at https://www.youtube.com/watch? v=BpHixKs91i8.

Details

IROS Conference 2020 Conference Paper

DMLO: Deep Matching LiDAR Odometry

Zhichao Li
Naiyan Wang

LiDAR odometry is a fundamental task for various areas such as robotics, autonomous driving. This problem is difficult since it requires the systems to be highly robust running in noisy real-world data. Existing methods are mostly local iterative methods. Feature-based global registration methods are not preferred since extracting accurate matching pairs in the nonuniform and sparse LiDAR data remains challenging. In this paper, we present Deep Matching LiDAR Odometry (DMLO), a novel learning-based framework which makes the feature matching method applicable to LiDAR odometry task. Unlike many recent learning-based methods, DMLO explicitly enforces geometry constraints in the framework. Specifically, DMLO decomposes the 6-DoF pose estimation into two parts, a learning-based matching network which provides accurate correspondences between two scans and rigid transformation estimation with a close-formed solution by Singular Value Decomposition (SVD). Comprehensive experimental results on real-world datasets KITTI and Argoverse demonstrate that our DMLO dramatically outperforms existing learning-based methods and is comparable with the state-of-the-art geometry- based approaches.

Details

ICRA Conference 2020 Conference Paper

Fast and Safe Path-Following Control using a State-Dependent Directional Metric

Zhichao Li
Ömür Arslan
Nikolay Atanasov 0001

This paper considers the problem of fast and safe autonomous navigation in partially known environments. Our main contribution is a control policy design based on ellipsoidal trajectory bounds obtained from a quadratic state-dependent distance metric. The ellipsoidal bounds are used to embed directional preference in the control design, leading to system behavior that is adapted to local environment geometry, carefully considering medial obstacles while paying less attention to lateral ones. We use a virtual reference governor system to adaptively follow a desired navigation path, slowing down when system safety may be violated and speeding up otherwise. The resulting controller is able to navigate complex environments faster than common Euclidean-norm and Lyapunov-function-based designs, while retaining stability and collision avoidance guarantees.

Details

NeurIPS Conference 2019 Conference Paper

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Jiawang Bian
Zhichao Li
Naiyan Wang
Huangying Zhan
Chunhua Shen
Ming-Ming Cheng
Ian Reid

Recent work has shown that CNN-based depth and ego-motion estimators can be learned using unlabelled monocular videos. However, the performance is limited by unidentified moving objects that violate the underlying static scene assumption in geometric image reconstruction. More significantly, due to lack of proper constraints, networks output scale-inconsistent results over different samples, i. e. , the ego-motion network cannot provide full camera trajectories over a long video sequence because of the per-frame scale ambiguity. This paper tackles these challenges by proposing a geometry consistency loss for scale-consistent predictions and an induced self-discovered mask for handling moving objects and occlusions. Since we do not leverage multi-task learning like recent works, our framework is much simpler and more efficient. Comprehensive evaluation results demonstrate that our depth estimator achieves the state-of-the-art performance on the KITTI dataset. Moreover, we show that our ego-motion network is able to predict a globally scale-consistent camera trajectory for long video sequences, and the resulting visual odometry accuracy is competitive with the recent model that is trained using stereo videos. To the best of our knowledge, this is the first work to show that deep networks trained using unlabelled monocular videos can predict globally scale-consistent camera trajectories over a long video sequence.

PDF Details