Author name cluster

Bingbing Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

ICRA Conference 2025 Conference Paper

AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

Mustafa Khan
Hamidreza Fazlali
Dhruv Sharma
Tongtong Cao
Dongfeng Bai
Yuan Ren
Bingbing Liu

Realistic scene reconstruction and view synthesis are essential for advancing autonomous driving systems by simulating safety-critical scenarios. 3D Gaussian Splatting (3DGS) excels in real-time rendering and static scene reconstructions but struggles with modeling driving scenarios due to complex backgrounds, dynamic objects, and sparse camera views. We propose AutoSplat, a framework employing Gaussian splatting to realistically reconstruct autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, our method enables multi-view consistent simulation of challenging scenarios, including lane changes. Leveraging 3D templates, we introduce a reflected Gaussian consistency constraint to supervise both the visible and unseen side of foreground objects. Moreover, to model the dynamic appearance of foreground objects, we estimate temporally-dependent residual spherical harmonics for each foreground Gaussian. Extensive experiments on Pandaset [1] and KITTI [2] demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. Our project page can be found here: https://autosplat.github.io/

ICLR Conference 2025 Conference Paper

Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction

Jing He
Haodong Li
Wei Yin 0006
Yixun Liang
Leheng Li
Kaiqiang Zhou
Hongbo Zhang
Bingbing Liu

Leveraging the visual priors of pre-trained text-to-image diffusion models offers a promising solution to enhance zero-shot generalization in dense prediction tasks. However, existing methods often uncritically use the original diffusion formulation, which may not be optimal due to the fundamental differences between dense prediction and image generation. In this paper, we provide a systemic analysis of the diffusion formulation for the dense prediction, focusing on both quality and efficiency. And we find that the original parameterization type for image generation, which learns to predict noise, is harmful for dense prediction; the multi-step noising/denoising diffusion process is also unnecessary and challenging to optimize. Based on these insights, we introduce $\textbf{Lotus}$, a diffusion-based visual foundation model with a simple yet effective adaptation protocol for dense prediction. Specifically, Lotus is trained to directly predict annotations instead of noise, thereby avoiding harmful variance. We also reformulate the diffusion process into a single-step procedure, simplifying optimization and significantly boosting inference speed. Additionally, we introduce a novel tuning strategy called detail preserver, which achieves more accurate and fine-grained predictions. Without scaling up the training data or model capacity, Lotus achieves SoTA performance in zero-shot depth and normal estimation across various datasets. It also enhances efficiency, being significantly faster than most existing diffusion-based methods. Lotus' superior quality and efficiency also enable a wide range of practical applications, such as joint estimation, single/multi-view 3D reconstruction, etc.

ICRA Conference 2025 Conference Paper

Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models

Tianshuo Xu
Hao Lu 0009
Xu Yan 0005
Yingjie Cai
Bingbing Liu
Ying-Cong Chen

Large Language Models (LLMs) have made substantial advancements in the field of robotic and autonomous driving. This study presents the first Occupancy-based Large Language Model (Occ-LLM), which represents a pioneering effort to integrate LLMs with an important representation. To effectively encode occupancy as input for the LLM and address the category imbalances associated with occupancy, we propose Motion Separation Variational Autoencoder (MS-VAE). This innovative approach utilizes prior knowledge to distinguish dynamic objects from static scenes before inputting them into a tailored Variational Autoencoder (VAE). This separation enhances the model's capacity to concentrate on dynamic trajectories while effectively reconstructing static scenes. The efficacy of Occ-LLM has been validated across key tasks, including 4D occupancy forecasting, self-ego planning, and occupancybased scene question answering. Comprehensive evaluations demonstrate that Occ-LLM significantly surpasses existing state-of-the-art methodologies, achieving gains of about 6% in Intersection over Union (IoU) and 4% in mean Intersection over Union (mIoU) for the task of 4D occupancy forecasting. These findings highlight the transformative potential of Occ-LLM in reshaping current paradigms within robotic and autonomous driving.

NeurIPS Conference 2025 Conference Paper

Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving

Yi Huang
Zhan Qu
Lihui Jiang
Bingbing Liu
Hongbo Zhang

End-to-end autonomous driving systems, predominantly trained through imitation learning, have demonstrated considerable effectiveness in leveraging large-scale expert driving data. Despite their success in open-loop evaluations, these systems often exhibit significant performance degradation in closed-loop scenarios due to causal confusion. This confusion is fundamentally exacerbated by the overreliance of the imitation learning paradigm on expert trajectories, which often contain unattributable noise and interfere with the modeling of causal relationships between environmental contexts and appropriate driving actions. To address this fundamental limitation, we propose Perception-Guided Self-Supervision (PGS)—a simple yet effective training paradigm that leverages perception outputs as the primary supervisory signals, explicitly modeling causal relationships in decision-making. The proposed framework aligns both the inputs and outputs of the decision-making module with perception results—such as lane centerlines and the predicted motions of surrounding agents—by introducing positive and negative self-supervision for the ego trajectory. This alignment is specifically designed to mitigate causal confusion arising from the inherent noise in expert trajectories. Equipped with perception-driven supervision, our method—built on a standard end-to-end architecture—achieves a Driving Score of 78. 08 and a mean success rate of 48. 64\% on the challenging closed-loop Bench2Drive benchmark, significantly outperforming existing state-of-the-art methods, including those employing more complex network architectures and inference pipelines. These results underscore the effectiveness and robustness of the proposed PGS framework, and point to a promising direction for addressing causal confusion and enhancing real-world generalization in autonomous driving.

NeurIPS Conference 2025 Conference Paper

SQS: Enhancing Sparse Perception Models via Query-based Splatting in Autonomous Driving

Haiming Zhang
Yiyao Zhu
Wending Zhou
Xu Yan
Yingjie Cai
Bingbing Liu
Shuguang Cui
Zhen Li

Sparse Perception Models (SPMs) adopt a query-driven paradigm that forgoes explicit dense BEV or volumetric construction, enabling highly efficient computation and accelerated inference. In this paper, we introduce SQS, a novel query-based splatting pre-training specifically designed to advance SPMs in autonomous driving. SQS introduces a plug-in module that predicts 3D Gaussian representations from sparse queries during pre-training, leveraging self-supervised splatting to learn fine-grained contextual features through the reconstruction of multi-view images and depth maps. During fine-tuning, the pre-trained Gaussian queries are seamlessly integrated into downstream networks via query interaction mechanisms that explicitly connect pre-trained queries with task-specific queries, effectively accommodating the diverse requirements of occupancy prediction and 3D object detection. Extensive experiments on autonomous driving benchmarks demonstrate that SQS delivers considerable performance gains across multiple query-based 3D perception tasks, notably in occupancy prediction and 3D object detection, outperforming prior state-of-the-art pre-training approaches by a significant margin (i. e. , +1. 3 mIoU on occupancy prediction and +1. 0 NDS on 3D detection).

AAAI Conference 2024 Conference Paper

RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation

Haiming Zhang
Xu Yan
Dongfeng Bai
Jiantao Gao
Pan Wang
Bingbing Liu
Shuguang Cui
Zhen Li

3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. However, image-based scene perception encounters significant challenges in achieving accurate prediction due to the absence of geometric priors. In this paper, we address this issue by exploring cross-modal knowledge distillation in this task, i.e., we leverage a stronger multi-modal model to guide the visual model during training. In practice, we observe that directly applying features or logits alignment, proposed and widely used in bird's-eye-view (BEV) perception, does not yield satisfactory results. To overcome this problem, we introduce RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction. By employing differentiable volume rendering, we generate depth and semantic maps in perspective views and propose two novel consistency criteria between the rendered outputs of teacher and student models. Specifically, the depth consistency loss aligns the termination distributions of the rendered rays, while the semantic consistency loss mimics the intra-segment similarity guided by vision foundation models (VLMs). Experimental results on the nuScenes dataset demonstrate the effectiveness of our proposed method in improving various 3D occupancy prediction approaches, e.g., our proposed methodology enhances our baseline by 2.2% in the metric of mIoU and achieves 50% in Occ3D benchmark.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Uplifting Range-View-based 3D Semantic Segmentation in Real-Time with Multi-Sensor Fusion

Shiqi Tan
Hamidreza Fazlali
Yixuan Xu
Yuan Ren
Bingbing Liu

Range-View(RV)-based 3D point cloud segmentation is widely adopted due to its compact data form. However, RV-based methods fall short in providing robust segmentation for the occluded points and suffer from distortion of projected RGB images due to the sparse nature of 3D point clouds. To alleviate these problems, we propose a new LiDAR and Camera Range-view-based 3D point cloud semantic segmentation method (LaCRange). Specifically, a distortion-compensating knowledge distillation (DCKD) strategy is designed to remedy the adverse effect of RV projection of RGB images. Moreover, a context-based feature fusion module is introduced for robust and preservative sensor fusion. Finally, in order to address the limited resolution of RV and its insufficiency of 3D topology, a new point refinement scheme is devised for proper aggregation of features in 2D and augmentation of point features in 3D. We evaluated the proposed method on large-scale autonomous driving datasets i. e. SemanticKITTI and nuScenes. In addition to being real-time, the proposed method achieves state-of-the-art results on nuScenes benchmark.

ICRA Conference 2023 Conference Paper

CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds

Enxu Li
Ryan Razani
Yixuan Xu
Bingbing Liu

A fast and accurate panoptic segmentation system for LiDAR point clouds is crucial for autonomous driving vehicles to understand the surrounding objects and scenes. Existing approaches usually rely on proposals or clustering to segment foreground instances. As a result, they struggle to achieve real-time performance. In this paper, we propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg. In particular, CPSeg comprises a shared encoder, a dual-decoder, and a cluster-free instance segmentation head, which is able to dynamically pillarize foreground points according to the learned embedding. Then, it acquires instance labels by finding connected pillars with a pairwise embedding comparison. Thus, the conventional proposal-based or clustering-based instance segmentation is transformed into a binary segmentation problem on the pairwise embedding comparison matrix. To help the network regress instance embedding, a fast and deterministic depth completion algorithm is proposed to calculate the surface normal of each point cloud in real-time. The proposed method is benchmarked on two large-scale autonomous driving datasets: SemanticKITTI and nuScenes. Notably, extensive experimental results show that CPSeg achieves state-of-the-art results among real-time approaches on both datasets.

ICRA Conference 2023 Conference Paper

PCGen: Point Cloud Generator for LiDAR Simulation

Chenqi Li
Yuan Ren
Bingbing Liu

Data is a fundamental building block for LiDAR perception systems. Unfortunately, real-world data collection and annotation is extremely costly & laborious. Recently, real data based LiDAR simulators have shown tremendous potential to complement real data, due to their scalability and high-fidelity compared to graphics engine based methods. Before simulation can be deployed in the real-world, two shortcomings need to be addressed. First, existing methods usually generate data which are more noisy and complete than the real point clouds, due to 3D reconstruction error and pure geometry-based raycasting method. Second, prior works on simulation for object detection focus solely on rigid objects, like cars, but Vulnerable Road User (VRU)s, like pedestrians, are important road participants. To tackle the first challenge, we propose First Peak Averaging (FPA) raycasting and surrogate model raydrop. FPA enables the simulation of both point cloud coordinates and sensor features, while taking into account reconstruction noise. The ray-wise surrogate raydrop model mimics the physical properties of LiDAR's laser receiver to determine whether a simulated point would be recorded by a real LiDAR. With minimal training data, the surrogate model can generalize to different geographies and scenes, closing the domain gap between raycasted and real point clouds. To tackle the simulation of deformable VRU simulation, we employ Skinned Multi-Person Linear model (SMPL) dataset to provide a pedestrian simulation baseline and compare the domain gap between CAD and reconstructed objects. Applying our pipeline to perform novel sensor synthesis, results show that object detection models trained by simulation data can achieve similar result as the real data trained model.

ICRA Conference 2022 Conference Paper

How to Build a Curb Dataset with LiDAR Data for Autonomous Driving

Dongfeng Bai
Tongtong Cao
Jingming Guo
Bingbing Liu

Curbs are one of the essential elements of urban and highway traffic environments. Robust curb detection provides road structure information for motion planning in an autonomous driving system. Commonly, video cameras and 3D LiDARs are mounted on autonomous vehicles for curb detection. However, camera-based methods suffer from challenging illumination conditions. During the long period of time before wide application of Deep Neural Network (DNN) with point clouds, LiDAR-based curb detection methods are based on hand-crafted features, which suffer from poor detection in some complex scenes. Recently, DNN-based dynamic object detection using LiDAR data has become prevalent, while few works pay attention to curb detection with a DNN approach due to lack of labeled data. A dataset with curb annotations or an efficient curb labeling approach, hence, is of high demand. In this paper, we present how to build a curb dataset with LiDAR data for autonomous driving highly automatically. Firstly, a Semantic High Definition map (SHD map) in a global coordinate frame is generated by applying both SLAM and semantic segmentation on consecutive LiDAR frames. Next, a Road HD map (RHD map) is generated from the SHD map by removing its dynamic noise caused by road users e. g. cars. After that, a Curb Instance map (CI map) can be obtained from the filtered RHD map by a series of curb point extraction and growing. Finally, the CI map can be projected back to single frames for direct, highly automatic curb labeling. In order to validate our proposed labeling method, on top of an open public LiDAR semantic dataset SemanticKITTI [1], an additional curb dataset is built. We run both semantic segmentation and instance segmentation methods on this built dataset. Experimental results show that the curb annotations have good consistency and accuracy. We released this dataset and it is publicly available at https://download.mindspore.cn.

ICRA Conference 2022 Conference Paper

Object Insertion Based Data Augmentation for Semantic Segmentation

Yuan Ren
Siyan Zhao
Bingbing Liu

Neural network used for the LiDAR semantic segmentation task needs the point-wise labeled point clouds for training, which is more expensive than bounding box annotations. Enhancing the diversity of training data through object insertion is an effective method to reduce labeling costs. The existing object insertion methods are mainly divided into two categories. First is “copy” the clusters from a LiDAR frame and “paste” it to other frames or positions. Second is inserting CAD models into the background then using LiDAR simulator to generate laser points of the inserted CAD models. “Copy-paste” method cannot generate realistic scanning lines and shadows, and the CAD models, especially the CAD models of flexible objects, are hard to obtain. We propose an object insertion based data augmentation method which can increase the performance of the semantic segmentation network remarkably. First, an object library is created by using the labeled LiDAR point clouds. Then, these objects are inserted into the LiDAR point clouds dynamically during the training. Finally, the realistic scanning lines and shadows are simulated according to the real LiDAR parameters. The experimental results show that the proposed augmentation method can increase the performance of different semantic segmentation frameworks remarkably.

ICRA Conference 2022 Conference Paper

SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering

Enxu Li
Ryan Razani
Yixuan Xu
Bingbing Liu

Panoptic segmentation aims to address semantic and instance segmentation simultaneously in a unified framework. However, an efficient solution of panoptic segmentation in applications like autonomous driving is still an open research problem. In this work, we propose a novel LiDAR-based panoptic system, called SMAC-Seg. We present a learnable sparse multi-directional attention clustering to segment multi-scale foreground instances. SMAC-Seg is a real-time clustering-based approach, which removes the complex proposal network to segment instances. Most existing clustering-based methods use the difference of the predicted and ground truth center offset as the only loss to supervise the instance centroid regression. However, this loss function only considers the centroid of the current object, but its relative position with respect to the neighbouring objects is not considered when learning to cluster. Thus, we propose to use a novel centroid-aware repel loss as an additional term to effectively supervise the network in order to differentiate each object cluster with its neighbours. Our experimental results show that SMAC-Seg achieves state-of-the-art performance among all real-time deployable networks on both large-scale public SemanticKITTI and nuScenes panoptic segmentation datasets.

ICRA Conference 2022 Conference Paper

Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters

Mrigank Rochan
Shubhra Aich
Eduardo R. Corral-Soto
Amir Nabatchian
Bingbing Liu

In this paper, we focus on a less explored, but more realistic and complex problem of domain adaptation in LiDAR semantic segmentation. There is a significant drop in performance of an existing segmentation model when training (source domain) and testing (target domain) data originate from different LiDAR sensors. To overcome this shortcoming, we propose an unsupervised domain adaptation framework that leverages unlabeled target domain data for self-supervision, coupled with an unpaired mask transfer strategy to mitigate the impact of domain shifts. Furthermore, we introduce the gated adapter module with a small number of parameters into the network to account for target domain-specific information. Experiments adapting from both real-to-real and synthetic-to-real LiDAR semantic segmentation benchmarks demonstrate the significant improvement over prior arts.

ICRA Conference 2021 Conference Paper

Bidirectional Attention Network for Monocular Depth Estimation

Shubhra Aich
Jean Marie Uwabeza Vianney
Md. Amirul Islam
Mannat Kaur
Bingbing Liu

In this paper, we propose a Bidirectional Attention Network (BANet), an end-to-end framework for monocular depth estimation (MDE) that addresses the limitation of effectively integrating local and global information in convolutional neural networks. The structure of this mechanism derives from a strong conceptual foundation of neural machine translation, and presents a light-weight mechanism for adaptive control of computation similar to the dynamic nature of recurrent neural networks. We introduce bidirectional attention modules that utilize the feed-forward feature maps and incorporate the global context to filter out ambiguity. Extensive experiments reveal the high degree of capability of this bidirectional attention model over feed-forward baselines and other state-of-the-art methods for monocular depth estimation on two challenging datasets - KITTI and DIODE. We show that our proposed approach either outperforms or performs at least on a par with the state-of-the-art monocular depth estimation methods with less memory and computational complexity.

ICRA Conference 2021 Conference Paper

LiDAR few-shot domain adaptation via integrated CycleGAN and 3D object detector with joint learning delay

Eduardo R. Corral-Soto
Amir Nabatchian
Martin Gerdzhev
Bingbing Liu

he success of supervised LiDAR perception methods relies on the availability of large sets of labeled point cloud data, for which the labeling process is costly and time consuming. Given unpaired LiDAR datasets of similar sizes from two domains, with one (source) containing task-specific labels e. g. 3D bounding boxes for all frames, but only a small percentage of frames being labeled in the other (target) domain, it is challenging to train a model that generalizes well on validation data from the target domain. In this paper we propose a novel LiDAR few-shot domain adaptation architecture and training strategy to address this challenge. Our method is based on adapting a task-specific network (3D object detector) to work within the CycleGAN framework modified to operate with LiDAR features, and on the joint end-to-end training of generators, discriminators, and task-specific layers. To overcome nonconvergence issues we propose a training strategy that introduces a mechanism to delay the joint learning between the generators/discriminators and the task-specific network by allowing them to start learning independently, while slowly introducing joint learning as they converge, hence avoiding instability during the early stages of the training. Our proposed integrated architecture enables a direct way to evaluate the performance of the model instead of feeding pre-computed generated data into a separate pretrained model. We include an experimental section where we evaluate our proposed architecture on the publicly available KITTI and Nuscenes datasets, as well as on our own labeled dataset. We present useful mean average precision plots that illustrate the benefits of our domain adaptation architecture as a function of number of labeled target domain frames.

ICRA Conference 2021 Conference Paper

Lite-HDSeg: LiDAR Semantic Segmentation Using Lite Harmonic Dense Convolutions

Ryan Razani
Ran Cheng
Ehsan Taghavi
Bingbing Liu

Autonomous driving vehicles and robotic systems rely on accurate perception of their surroundings. Scene understanding is one of the crucial components of perception modules. Among all available sensors, LiDARs are one of the essential sensing modalities of autonomous driving systems due to their active sensing nature with high resolution of sensor readings. Accurate and fast semantic segmentation methods are needed to fully utilize LiDAR sensors for scene understanding. In this paper, we present Lite-HDSeg, a novel real-time convolutional neural network for semantic segmentation of full 3D LiDAR point clouds. Lite-HDSeg can achieve the best accuracy vs. computational complexity trade-off in SemanticKITTI bench-mark and is designed on the basis of a new encoder-decoder architecture with light-weight harmonic dense convolutions as its core. Moreover, we introduce ICM, an improved global contextual module to capture multi-scale contextual features, and MCSPN, a multi-class Spatial Propagation Network to further refine the semantic boundaries. Our experimental results show that the proposed method outperforms state-of- the-art semantic segmentation approaches which can run real-time, thus is suitable for robotic and autonomous driving applications.

ICRA Conference 2021 Conference Paper

S3Net: 3D LiDAR Sparse Semantic Segmentation Network

Ran Cheng
Ryan Razani
Yuan Ren
Bingbing Liu

Semantic Segmentation is a crucial component in the perception systems of many applications, such as robotics and autonomous driving that rely on accurate environmental perception and understanding. In literature, several approaches are introduced to attempt LiDAR semantic segmentation task, such as projection-based (range-view or birds-eye-view), and voxel-based approaches. However, they either abandon the valuable 3D topology and geometric relations and suffer from information loss introduced in the projection process or are inefficient. Therefore, there is a need for accurate models capable of processing the 3D driving-scene point cloud in 3D space. In this paper, we propose S3Net, a novel convolutional neural network for LiDAR point cloud semantic segmentation. It adopts an encoder-decoder backbone that consists of Sparse Intra-channel Attention Module (SIntraAM), and Sparse Inter-channel Attention Module (SInterAM) to emphasize the fine details of both within each feature map and among nearby feature maps. To extract the global contexts in deeper layers, we introduce Sparse Residual Tower based upon sparse convolution that suits varying sparsity of LiDAR point cloud. In addition, geo-aware anisotrophic loss is leveraged to emphasize the semantic boundaries and penalize the noise within each predicted regions, leading to a robust prediction. Our experimental results show that the proposed method leads to a large improvement (12%) compared to its baseline counterpart (MinkNet42 [1]) on SemanticKITTI [2] test set and achieves state-of-the-art mIoU accuracy of semantic segmentation approaches.

ICRA Conference 2021 Conference Paper

TORNADO-Net: mulTiview tOtal vaRiatioN semAntic segmentation with Diamond inceptiOn module

Martin Gerdzhev
Ryan Razani
Ehsan Taghavi
Bingbing Liu

Semantic segmentation of point clouds is a key component of scene understanding for robotics and autonomous driving. In this paper, we introduce TORNADO-Net - a neural network for 3D LiDAR point cloud semantic segmentation. We incorporate a multi-view (bird-eye and range) projection feature extraction with an encoder-decoder ResNet architecture with a novel diamond context block. Current projection-based methods do not take into account that neighboring points usually belong to the same class. To better utilize this local neighbourhood information and reduce noisy predictions, we introduce a combination of Total Variation, Lovász-Softmax, and Weighted Cross-Entropy losses. We also take advantage of the fact that the LiDAR data encompasses 360 ◦ field of view and use circular padding. We demonstrate state-of-the-art results on the SemanticKITTI dataset and also provide thorough quantitative evaluations and ablation results.

IROS Conference 2015 Conference Paper

B-SHOT: A binary feature descriptor for fast and efficient keypoint matching on 3D point clouds

Sai Manoj Prakhya
Bingbing Liu
Weisi Lin

In this paper, we introduce the very first ‘binary’ 3D feature descriptor, B-SHOT, for fast and efficient keypoint matching on 3D point clouds. We propose a binary quantization method that converts a real valued vector to a binary vector. We apply this method on a state-of-the-art 3D feature descriptor, SHOT [1], and create a new binary 3D feature descriptor. B-SHOT requires 32 times lesser memory for its representation while being 6 times faster in feature descriptor matching, when compared to the SHOT feature descriptor. Experimental evaluation shows that B-SHOT offers comparable keypoint matching performance to that of the state-of-the-art 3D feature descriptors on a standard benchmark dataset.

ICRA Conference 2015 Conference Paper

Sparse Depth Odometry: 3D keypoint based pose estimation from dense depth data

Sai Manoj Prakhya
Bingbing Liu
Weisi Lin
Usman Qayyum

This paper presents Sparse Depth Odometry (SDO) to incrementally estimate the 3D pose of a depth camera in indoor environments. SDO relies on 3D keypoints extracted on dense depth data and hence can be used to augment the RGB-D camera based visual odometry methods that fail in places where there is no proper illumination. In SDO, our main contribution is the design of the keypoint detection module, which plays a vital role as it condenses the input point cloud to a few keypoints. SDO differs from existing depth alone methods as it does not use the popular signed distance function and can run online, even without a GPU. A new keypoint detection module is proposed via keypoint selection, and is based on extensive theoretical and experimental evaluation. The proposed keypoint detection module comprises of two existing keypoint detectors, namely SURE [1] and NARF [2]. It offers reliable keypoints that describe the scene more comprehensively, compared to others. Finally, an extensive performance evaluation of SDO on benchmark datasets with the proposed keypoint detection module is presented and compared with the state-of-the-art.

ICRA Conference 2010 Conference Paper

DAvinCi: A cloud computing framework for service robots

Rajesh Arumugam
Vikas Reddy Enti
Bingbing Liu
Xiaojun Wu
Krishnamoorthy Baskaran
Foo Kong Foong
Appadorai Senthil Kumar
Dee Meng Kang

We propose DAvinCi, a software framework that provides the scalability and parallelism advantages of cloud computing for service robots in large environments. We have implemented such a system around the Hadoop cluster with ROS (Robotic Operating system) as the messaging framework for our robotic ecosystem. We explore the possibilities of parallelizing some of the robotics algorithms as Map/Reduce tasks in Hadoop. We implemented the FastSLAM algorithm in Map/Reduce and show how significant performance gains in execution times to build a map of a large area can be achieved with even a very small eight-node Hadoop cluster. The global map can later be shared with other robots introduced in the environment via a Software as a Service (SaaS) Model. This reduces the burden of exploration and map building for the new robot and minimizes it's need for additional sensors. Our primary goal is to develop a cloud computing environment which provides a compute cluster built with commodity hardware exposing a suite of robotic algorithms as a SaaS and share data co-operatively across the robotic ecosystem.

IROS Conference 2005 Conference Paper

Minima controlled recursive averaging noise reduction for multi-aided inertial navigation of ground vehicles

Bingbing Liu
Martin David Adams
Javier Ibañez-Guzmán

Low-cost inertial measurement units (IMUs) are increasingly becoming commercially available and the use of IMUs in autonomous vehicle applications has increased rapidly in the past decade. IMUs are subject to various errors, such as biases, drifts, nonlinearities, scale factors and noise. The noise is produced by various sources, such as thermal and vibrational disturbances. Noise estimation is critical in accurate inertial navigation systems (INS). The main contribution of this paper is that a noise analysis of the raw accelerations measured by IMUs during signal presence (i. e, an acceleration caused by a specific force) or absence (when the IMU undergoes constant velocity) is carried out to reduce these noise components, which corrupt the inertial data. After noise reduction, multi-aiding information from odometry, a single-axis gyroscope and vehicle constraints is utilized to bound the error growth of the inertial data and produce a reliable outdoor localization system. Experimental results are presented to show the effectiveness of the noise reduction method and the improved accuracy of the multi-aided INS.

ICRA Conference 2005 Conference Paper

Multi-aided Inertial Navigation for Ground Vehicles in Outdoor Uneven Environments

Bingbing Liu
Martin David Adams
Javier Ibañez-Guzmán

A good localization ability is essential for an autonomous vehicle to perform any functions. For ground vehicles operating in outdoor, uneven and unstructured environments, the localization task becomes much more difficult than in indoor environments. In urban or forest environments where high buildings or tall trees exist, GPS sensors also fail easily. The main contribution of this paper is that a multi-aided inertial based localization system has been developed to solve the outdoor localization problem. The multi-aiding information is from odometry, an accurate gyroscope and vehicle constraints. Contrary to previous work, a kinematic model is developed to estimate the inertial sensor’s lateral velocity. This is particularly important when cornering at speed, and side slip occurs. Experimental results are presented of this system which is able to provide a vehicle’s position, velocity and attitude estimation accurately, even when the testing vehicle runs in outdoor uneven environments.

IROS Conference 2004 Conference Paper

Range errors due to occlusion in non-coaxial LADARs

Bingbing Liu
Martin David Adams
Javier Ibañez-Guzmán
Wijerupage Sardha Wijesoma

A prerequisite for mobile robot navigation is a reliable sensing mechanism. Laser detection and ranging sensors or LADARs are widely used in mobile robotics. When processing LADAR data for the purposes of feature extraction and/or data association, most previous work models such device as processing range data which follows a normal distribution. In this paper, it would be demonstrated that commonly used LADARs suffer from incorrect range readings at range discontinuities, which can have a much more detrimental effect on feature extraction or data association algorithms than random noise. LADARs with separated transmitter and receiver configuration can introduce a significant occlusion effect, as the reflected laser energy from the target can be partially occluded from the receiver. This paper would demonstrate that false range values can result from LADARs and that the occurrence of these values can be reliably predicted by monitoring the received signal strength. A useful design criterion for the optical separation of the transmitter and receiver is also derived for non-coaxial LADARs. The parameters, which are related to range errors, are quantified so that such errors can be reliably predicted.