Author name cluster

Mark E. Campbell

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

39 papers

1 author row

ICLR Conference 2025 Conference Paper

Learning 3D Perception from Others' Predictions

Jinsu Yoo
Zhenyang Feng
Tai-Yu Pan
Yihong Sun
Cheng Perng Phoo
Xiangyu Chen 0007
Mark E. Campbell
Kilian Q. Weinberger

Accurate 3D object detection in real-world environments requires a huge amount of annotated data with high quality. Acquiring such data is tedious and expensive, and often needs repeated effort when a new sensor is adopted or when the detector is deployed in a new environment. We investigate a new scenario to construct 3D object detectors: *learning from the predictions of a nearby unit that is equipped with an accurate detector.* For example, when a self-driving car enters a new area, it may learn from other traffic participants whose detectors have been optimized for that area. This setting is label-efficient, sensor-agnostic, and communication-efficient: nearby units only need to share the predictions with the ego agent (e.g., car). Naively using the received predictions as ground-truths to train the detector for the ego car, however, leads to inferior performance. We systematically study the problem and identify viewpoint mismatches and mislocalization (due to synchronization and GPS errors) as the main causes, which unavoidably result in false positives, false negatives, and inaccurate pseudo labels. We propose a distance-based curriculum, first learning from closer units with similar viewpoints and subsequently improving the quality of other units' predictions via self-training. We further demonstrate that an effective pseudo label refinement module can be trained with a handful of annotated data, largely reducing the data quantity necessary to train an object detector. We validate our approach on the recently released real-world collaborative driving dataset, using reference cars' predictions as pseudo labels for the ego car. Extensive experiments including several scenarios (e.g., different sensors, detectors, and domains) demonstrate the effectiveness of our approach toward label-efficient learning of 3D perception from other units' predictions.

ICRA Conference 2025 Conference Paper

Robust Swimming Controller for Soft Robots via Drop-Out Learning

Josephine Monica
Mark E. Campbell

A novel framework for training a robotic fish to learn how to swim, even in the presence of degradations or failures in actuators is developed. Robotic underwater robots, particularly soft fish-inspired designs have gained significant attention due to their distinct benefits, including superior maneuverability, energy efficiency, versatile applications, and seamless integration with marine environments. However, their material properties and actuators can degrade, leading to pre-mature system failures. In this paper, we introduce the concept of actuator drop-out during training, to enable the robot to learn how to swim even when one or more actuators are degraded or non-functional. A Soft Actor-Critic Deep Reinforcement Learning architecture is used to learn a policy, with actuator degradations/failures introduced during training. A four actuator koi fish is modeled and simulated using the FishGym environment. Navigation-based validation tests show little degradation with one actuator failure, and much more robust swimming behaviors and performance compared to training with no failures, even when two or three actuators fail. These results will improve long-term operational reliability, ensuring robot fish functionality even in challenging underwater conditions.

ICRA Conference 2024 Conference Paper

Better Monocular 3D Detectors with LiDAR from the Past

Yurong You
Cheng Perng Phoo
Carlos Andres Diaz-Ruiz
Katie Luo
Wei-Lun Chao
Mark E. Campbell
Bharath Hariharan
Kilian Q. Weinberger

Accurate 3D object detection is crucial to autonomous driving. Though LiDAR-based detectors have achieved impressive performance, the high cost of LiDAR sensors precludes their widespread adoption in affordable vehicles. Camera-based detectors are cheaper alternatives but often suffer inferior performance compared to their LiDAR-based counterparts due to inherent depth ambiguities in images. In this work, we seek to improve monocular 3D detectors by leveraging unlabeled historical LiDAR data. Specifically, at inference time, we assume that the camera-based detectors have access to multiple unlabeled LiDAR scans from past traversals at locations of interest (potentially from other high-end vehicles equipped with LiDAR sensors). Under this setup, we proposed a novel, simple, and end-to-end trainable framework, termed AsyncDepth, to effectively extract relevant features from asynchronous LiDAR traversals of the same location for monocular 3D detectors. We show consistent and significant performance gain (up to 9 AP) across multiple state-of-the-art models and datasets with a negligible additional latency of 9. 66 ms and a small storage cost. Our code can be found at https://github.com/YurongYou/AsyncDepth.

ICLR Conference 2024 Conference Paper

Pre-training LiDAR-based 3D Object Detectors through Colorization

Tai-Yu Pan
Chenyang Ma
Tianle Chen
Cheng Perng Phoo
Katie Luo
Yurong You
Mark E. Campbell
Kilian Q. Weinberger

Accurate 3D object detection and understanding for self-driving cars heavily relies on LiDAR point clouds, necessitating large amounts of labeled data to train. In this work, we introduce an innovative pre-training approach, Grounded Point Colorization (GPC), to bridge the gap between data and labels by teaching the model to colorize LiDAR point clouds, equipping it with valuable semantic cues. To tackle challenges arising from color variations and selection bias, we incorporate color as "context" by providing ground-truth colors as hints during colorization. Experimental results on the KITTI and Waymo datasets demonstrate GPC's remarkable effectiveness. Even with limited labeled data, GPC significantly improves fine-tuning performance; notably, on just 20% of the KITTI dataset, GPC outperforms training from scratch with the entire dataset. In sum, we introduce a fresh perspective on pre-training for 3D object detection, aligning the objective with the model's intended role and ultimately advancing the accuracy and efficiency of 3D object detection for autonomous vehicles.

IROS Conference 2024 Conference Paper

SWIFT: Strategic Weather-informed Image-based Forecasting for Trajectories

Youya Xia
Jose Nino
Yutao Han
Mark E. Campbell

Predicting agents’ trajectories in complex environments is critical for achieving safe autonomous robot navigation. Empirically, agents’ decisions and preferences are susceptible to changes in environmental factors (e. g. , interactions with other agents, weather conditions, traffic rules). State-of-the-art methods rely on High-Definition (HD) or semantic maps to model the environment, but do not take into account unpredictable factors such as complex weather conditions. In addition, since HD maps are nontrivial to obtain, those methods are limited in the scope of environments they can be applied in. We propose a more flexible graph based trajectory prediction model that uses only images to model the environment, without requiring expensive map information. We experimentally validate our proposed model, demonstrating robust performances in trajectory prediction compared to state-of-the-art methods, and outperform in complex environments that cannot be modeled with purely map based methods, such as diverse weather conditions.

ICRA Conference 2023 Conference Paper

Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs

Youya Xia
Josephine Monica
Wei-Lun Chao
Bharath Hariharan
Kilian Q. Weinberger
Mark E. Campbell

A self-driving car must be able to reliably handle adverse weather conditions (e. g. , snowy) to operate safely. In this paper, we investigate the idea of turning sensor inputs (i. e. , images) captured in an adverse condition into a benign one (i. e. , sunny), upon which the downstream tasks (e. g. , semantic segmentation) can attain high accuracy. Prior work primarily formulates this as an unpaired image-to-image translation problem due to the lack of paired images captured under the exact same camera poses and semantic layouts. While perfectly-aligned images are not available, one can easily obtain coarsely-paired images. For instance, many people drive the same routes daily in both good and adverse weather; thus, images captured at close-by GPS locations can form a pair. Though data from repeated traversals are unlikely to capture the same foreground objects, we posit that they provide rich contextual information to supervise the image translation model. To this end, we propose a novel training objective leveraging coarsely-aligned image pairs. We show that our coarsely-aligned training scheme leads to a better image translation quality and improved downstream tasks, such as semantic segmentation, monocular depth estimation, and visual localization.

ICRA Conference 2023 Conference Paper

Probabilistic Uncertainty Quantification of Prediction Models with Application to Visual Localization

Junan Chen
Josephine Monica
Wei-Lun Chao
Mark E. Campbell

The uncertainty quantification of prediction models (e. g. , neural networks) is crucial for their adoption in many robotics applications. This is arguably as important as making accurate predictions, especially for safety-critical applications such as self-driving cars. This paper proposes our approach to uncertainty quantification in the context of visual localization for autonomous driving, where we predict locations from images. Our proposed framework estimates probabilistic uncertainty by creating a sensor error model that maps an internal output of the prediction model to the uncertainty. The sensor error model is created using multiple image databases of visual localization, each with ground-truth location. We demonstrate the accuracy of our uncertainty prediction framework using the Ithaca365 dataset, which includes variations in lighting, weather (sunny, snowy, night), and alignment errors between databases. We analyze both the predicted uncertainty and its incorporation into a Kalman-based localization filter. Our results show that prediction error variations increase with poor weather and lighting condition, leading to greater uncertainty and outliers, which can be predicted by our proposed uncertainty model. Additionally, our probabilistic error model enables the filter to remove ad hoc sensor gating, as the uncertainty automatically adjusts the model to the input data.

ICRA Conference 2022 Conference Paper

Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection in Self-Driving Cars

Yurong You
Carlos Andres Diaz-Ruiz
Yan Wang 0051
Wei-Lun Chao
Bharath Hariharan
Mark E. Campbell
Kilian Q. Weinberger

Self-driving cars must detect other traffic participants like vehicles and pedestrians in 3D in order to plan safe routes and avoid collisions. State-of-the-art 3D object detectors, based on deep learning, have shown promising accuracy but are prone to over-fit domain idiosyncrasies, making them fail in new environments-a serious problem for the robustness of self-driving cars. In this paper, we propose a novel learning approach that reduces this gap by fine-tuning the detector on high-quality pseudo-labels in the target domain - pseudo-labels that are automatically generated after driving based on replays of previously recorded driving sequences. In these replays, object tracks are smoothed forward and backward in time, and detections are interpolated and extrapolated-crucially, leveraging future information to catch hard cases such as missed detections due to occlusions or far ranges. We show, across five autonomous driving datasets, that fine-tuning the object detector on these pseudo-labels substantially reduces the domain gap to new driving environments, yielding strong improvements detection reliability and accuracy.

ICLR Conference 2022 Conference Paper

Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception

Yurong You
Katie Luo
Xiangyu Chen 0007
Junan Chen
Wei-Lun Chao
Wen Sun 0002
Bharath Hariharan
Mark E. Campbell

Self-driving cars must detect vehicles, pedestrians, and other trafﬁc participants accurately to operate safely. Small, far-away, or highly occluded objects are particularly challenging because there is limited information in the LiDAR point clouds for detecting them. To address this challenge, we leverage valuable information from the past: in particular, data collected in past traversals of the same scene. We posit that these past data, which are typically discarded, provide rich contextual information for disambiguating the above-mentioned challenging cases. To this end, we propose a novel end-to-end trainable Hindsight framework to extract this contextual information from past traversals and store it in an easy-to-query data structure, which can then be leveraged to aid future 3D object detection of the same scene. We show that this framework is compatible with most modern 3D detection architectures and can substantially improve their average precision on multiple autonomous driving datasets, most notably by more than 300% on the challenging cases. Our code is available at https://github.com/YurongYou/Hindsight.

ICRA Conference 2022 Conference Paper

Is it Worth to Reason about Uncertainty in Occupancy Grid Maps during Path Planning?

Jacopo Banfi
Lindsey Woo
Mark E. Campbell

This paper investigates the usefulness of reasoning about the uncertain presence of obstacles during path planning, which typically stems from the usage of probabilistic occupancy grid maps for representing the environment when mapping via a noisy sensor like a stereo camera. The traditional planning paradigm prescribes using a hard threshold on the occupancy probability to declare that a cell is an obstacle, and to plan a single path accordingly while treating unknown space as free. We compare this approach against a new uncertainty-aware planner, which plans two different path hypotheses and then merges their initial trajectory segments into a single one ending in a “next-best view” pose. After this informative view is taken, the planner commits to one of the hypotheses, or to a completely new one if a collision is imminent. Simulations were conducted comparing the proposed and traditional planner. Results show the existence of planning scenarios -like when the environment contains a dead-end, or when the goal is placed close to an obstacle- in which reasoning about uncertainty can significantly decrease the robot's traveled distance and increase the chances of reaching the goal. The new planner was also validated on a real Clearpath Jackal robot equipped with a ZED 2 stereo camera.

IROS Conference 2022 Conference Paper

Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments

Vikram Shree
Sarah Allen
Beatriz A. Asfora
Jacopo Banfi
Mark E. Campbell

There has been a plethora of work towards im-proving robot perception and navigation, yet their application in hazardous environments, like during a fire or an earthquake, is still at a nascent stage. We hypothesize two key challenges here: first, it is difficult to replicate such scenarios in the real world, which is necessary for training and testing purposes. Second, current systems are not fully able to take advantage of the rich multi-modal data available in such hazardous environments. To address the first challenge, we propose to harness the enormous amount of visual content available in the form of movies and TV shows, and develop a dataset that can represent hazardous environments encountered in the real world. The data is annotated with high-level danger ratings for realistic disaster images, and corresponding keywords are provided that summarize the content of the scene. In response to the second challenge, we propose a multi-modal danger estimation pipeline for collaborative human-robot escape scenarios. Our Bayesian framework improves danger estimation by fusing information from robot's camera sensor and language inputs from the human. Furthermore, we augment the estimation module with a risk-aware planner that helps in identifying safer paths out of the dangerous environment. Through extensive simulations, we exhibit the advantages of our multi-modal perception framework that gets translated into tangible benefits such as higher success rate in a collaborative human-robot mission.

ICRA Conference 2022 Conference Paper

Sequential Joint Shape and Pose Estimation of Vehicles with Application to Automatic Amodal Segmentation Labeling

Josephine Monica
Wei-Lun Chao
Mark E. Campbell

Shape and pose estimation is a critical perception problem for a self-driving car to fully understand its surrounding environment. One fundamental challenge in solving this problem is the incomplete sensor signal (e. g. , LiDAR scans), especially for faraway or occluded objects. In this paper, we propose a novel algorithm to address this challenge, which explicitly leverages the sensor signal captured over consecutive time: the consecutive signals can provide more information about an object, including different viewpoints and its motion. By encoding the consecutive signals via a recurrent neural network, not only our algorithm improves the shape and pose estimates, but also produces a labeling tool that can benefit other tasks in autonomous driving research. Specifically, building upon our algorithm, we propose a novel pipeline to automatically annotate high-quality labels for amodal segmentation on images, which are hard and laborious to annotate manually. Our code and data will be made publicly available.

ICRA Conference 2021 Conference Paper

Detecting and Mapping Trees in Unstructured Environments with a Stereo Camera and Pseudo-Lidar

Brian H. Wang
Carlos Andres Diaz-Ruiz
Jacopo Banfi
Mark E. Campbell

We present a method for detecting and mapping trees in noisy stereo camera point clouds, using a learned 3D object detector. Inspired by recent advancements in 3-D object detection using a pseudo-lidar representation for stereo data, we train a PointRCNN detector to recognize trees in forest-like environments. We generate detector training data with a novel automatic labeling process that clusters a fused global point cloud. This process annotates large stereo point cloud training data sets with minimal user supervision, and unlike previous pseudo-lidar detection pipelines, requires no 3D ground truth from other sensors such as lidar. Our mapping system additionally uses a Kalman filter to associate detections and consistently estimate the positions and sizes of trees. We collect a data set for tree detection consisting of 8680 stereo point clouds, and validate our method on an outdoors test sequence. Our results demonstrate robust tree recognition in noisy stereo data at ranges of up to 7 meters, on 720p resolution images from a Stereolabs ZED 2 camera. Code and data are available at https://github.com/brian-h-wang/pseudolidar-tree-detection.

ICRA Conference 2020 Conference Paper

DeepSemanticHPPC: Hypothesis-based Planning over Uncertain Semantic Point Clouds

Yutao Han
Hubert Lin
Jacopo Banfi
Kavita Bala
Mark E. Campbell

Planning in unstructured environments is challenging - it relies on sensing, perception, scene reconstruction, and reasoning about various uncertainties. We propose DeepSemanticHPPC, a novel uncertainty-aware hypothesis-based planner for unstructured environments. Our algorithmic pipeline consists of: a deep Bayesian neural network which segments surfaces with uncertainty estimates; a flexible point cloud scene representation; a next-best-view planner which minimizes the uncertainty of scene semantics using sparse visual measurements; and a hypothesis-based path planner that proposes multiple kinematically feasible paths with evolving safety confidences given next-best-view measurements. Our pipeline iteratively decreases semantic uncertainty along planned paths, filtering out unsafe paths with high confidence. We show that our framework plans safe paths in real-world environments where existing path planners typically fail.

ICLR Conference 2020 Conference Paper

Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving

Yurong You
Yan Wang 0051
Wei-Lun Chao
Divyansh Garg
Geoff Pleiss
Bharath Hariharan
Mark E. Campbell
Kilian Q. Weinberger

Detecting objects such as cars and pedestrians in 3D plays an indispensable role in autonomous driving. Existing approaches largely rely on expensive LiDAR sensors for accurate depth information. While recently pseudo-LiDAR has been introduced as a promising alternative, at a much lower cost based solely on stereo images, there is still a notable performance gap. In this paper we provide substantial advances to the pseudo-LiDAR framework through improvements in stereo depth estimation. Concretely, we adapt the stereo network architecture and loss function to be more aligned with accurate depth estimation of faraway objects --- currently the primary weakness of pseudo-LiDAR. Further, we explore the idea to leverage cheaper but extremely sparse LiDAR sensors, which alone provide insufficient information for 3D detection, to de-bias our depth estimation. We propose a depth-propagation algorithm, guided by the initial depth estimates, to diffuse these few exact measurements across the entire depth map. We show on the KITTI object detection benchmark that our combined approach yields substantial improvements in depth estimation and stereo-based 3D object detection --- outperforming the previous state-of-the-art detection accuracy for faraway objects by 40%. Our code is available at https://github.com/mileyan/Pseudo_Lidar_V2.

IROS Conference 2020 Conference Paper

Vision Only 3-D Shape Estimation for Autonomous Driving

Josephine Monica
Mark E. Campbell

We present a probabilistic framework for detailed 3-D shape estimation and tracking using only vision measurements. Vision detections are processed via a bird's eye view representation, creating accurate detections at far ranges. A probabilistic model of the vision based point cloud measurements is learned and used in the framework. A 3-D shape model is developed by fusing a set of point cloud detections via a recursive Best Linear Unbiased Estimator (BLUE). The point cloud fusion accounts for noisy and inaccurate measurements, as well as minimizing growth of points in the 3-D shape. The use of a tracking algorithm and sensor pose enables 3-D shape estimation of dynamic objects from a moving car. Results are analyzed on experimental data, demonstrating the ability of our approach to produce more accurate and cleaner shape estimates.

ICRA Conference 2019 Conference Paper

Anytime Stereo Image Depth Estimation on Mobile Devices

Yan Wang 0051
Zihang Lai
Gao Huang 0001
Brian H. Wang
Laurens van der Maaten
Mark E. Campbell
Kilian Q. Weinberger

Many applications of stereo depth estimation in robotics require the generation of accurate disparity maps in real time under significant computational constraints. Current state-of-the-art algorithms force a choice between either generating accurate mappings at a slow pace, or quickly generating inaccurate ones, and additionally these methods typically require far too many parameters to be usable on power- or memory-constrained devices. Motivated by these shortcomings, we propose a novel approach for disparity prediction in the anytime setting. In contrast to prior work, our end-to-end learned approach can trade off computation and accuracy at inference time. Depth estimation is performed in stages, during which the model can be queried at any time to output its current best estimate. Our final model can process 1242×375 resolution images within a range of 10-35 FPS on an NVIDIA Jetson TX2 module with only marginal increases in error - using two orders of magnitude fewer parameters than the most competitive baseline. The source code is available at https://github.com/mileyan/AnyNet.

ICRA Conference 2018 Conference Paper

Perception-Informed Autonomous Environment Augmentation with Modular Robots

Tarik Tosun
Jonathan Daudelin
Gangyuan Jing
Hadas Kress-Gazit
Mark E. Campbell
Mark Yim

We present a system enabling a modular robot to autonomously build structures in order to accomplish high-level tasks. Building structures allows the robot to surmount large obstacles, expanding the set of tasks it can perform. This addresses a common weakness of modular robot systems, which often struggle to traverse large obstacles. This paper presents the hardware, perception, and planning tools that comprise our system. An environment characterization algorithm identifies features in the environment that can be augmented to create a path between two disconnected regions of the environment. Specially-designed building blocks enable the robot to create structures that can augment the environment to make obstacles traversable. A high-level planner reasons about the task, robot locomotion capabilities, and environment to decide if and where to augment the environment in order to perform the desired task. We validate our system in hardware experiments.

ICRA Conference 2016 Conference Paper

An efficient probabilistic surface normal estimator

Daniel J. Lee
Mark E. Campbell

An efficient surface normal estimation method is presented. The new algorithm estimates surface normal direction for each cell in a grid based on the occupancy information (both occupied and empty) of the neighboring cells. This grid representation allows user-defined sizes and scaling with the environment, not the number of measurements. Recursive and batch formulations to obtain the posterior estimate are presented, and compared. A computationally efficient implementation is derived which provides consistent and accurate estimates as measurements become available. Both simulation and experimental results are shown, demonstrating comparable estimation performance to that of using Point Cloud Library, but with significantly reduced computation time.

ICRA Conference 2016 Conference Paper

An efficient robotic exploration planner with probabilistic guarantees

Alexander Ivanov 0002
Mark E. Campbell

Efficient robotic exploration of an unknown, sensor limited, global-information-deficient environment poses a unique challenge to path planning algorithms because no deterministic guarantees on path completion and mission success can be made. Integrated Exploration (IE), which strives to combine localization and exploration, must be solved in order to create an autonomous robotic system capable of long term operation in new and challenging environments. This paper formulates a probabilistic framework which allows the creation of exploration algorithms providing probabilistic guarantees of success. A novel connection is made between the Hamiltonian Path Problem and exploration. The Guaranteed Probabilistic Information Explorer (G-PIE) is developed for the IE problem, providing a probabilistic guarantee on path completion, and asymptotic optimality of exploration.

ICRA Conference 2016 Conference Paper

Probabilistic qualitative mapping for robots

Jennifer Padgett
Mark E. Campbell

A probabilistic qualitative relational mapping (PQRM) algorithm is developed to enable robots to robustly map environments using noisy sensor measurements. Qualitative state representations provide soft, relative map information which is robust to metrical errors. In this paper, probabilistic distributions over qualitative states are derived and an algorithm to update the map recursively is developed. Maps are evaluated using Monte Carlo simulations for convergence and correctness. Validation tests are conducted on the New College dataset to evaluate map performance in realistic environments.

IROS Conference 2015 Conference Paper

Human-robot information sharing with structured language generation from probabilistic beliefs

Rina Tse
Mark E. Campbell

This paper presents a framework for information sharing and fusion in cooperative tasks involving humans and robots. In this context, all information regarding the state of interest is recursively fused and maintained by each agent in a form of belief. For a robot agent, its belief is commonly and practically represented as a probability density function (pdf), formed by traditional sensor fusion and state estimation algorithms. In cooperative tasks with non-expert humans, a robot needs to effectively communicate its belief so that the gathered information can be easily processed and interpreted by the humans. The goal of this research is to provide two-way information exchange and fusion between robots and humans, the former operating on pdfs, while the latter on English sentences. This is achieved by considering two goodness measures: semantic correctness and information preservation. Based on the goodness measures studied, results show that the proposed framework is able to generate optimal statements describing the given belief pdfs and successfully recover the initial inputs used to generate them. Additionally, in order to describe complex belief pdfs, a Mixture of Statements (MoS) model is proposed such that the optimal expression can be generated through a composition of more than one statements. With a nonparametric Dirichlet Process MoS generation, it is found that the robot can determine correctly the number of statements as well as the corresponding reference parameters needed to describe all hypotheses underlying its belief.

ICRA Conference 2015 Conference Paper

Joint tracking and non-parametric shape estimation of arbitrary extended objects

Kevin Wyffels
Mark E. Campbell

This paper presents a probabilistically rigorous method for jointly estimating the shape and kinematic states of arbitrary extended objects. A non-parametric shape model is defined as a set of points sampled from the object surface, and the joint probability density function over the surface samples is estimated recursively over time from lidar data. The presented work is demonstrated for a single maneuvering, non-convex object, highlighting key advantages over existing methods and motivating further development.

ICRA Conference 2013 Conference Paper

Modeling and fusing negative information for dynamic extended multi-object tracking

Kevin Wyffels
Mark E. Campbell

A novel approach to utilizing negative information to improve the accuracy of extended multi-object tracking is presented. The parameterized probability density of object tracks unresolved in sensor data is updated via inferences about the sensor-to-object geometries necessary to result in occlusion of the unresolved object. Negative information is also leveraged to improve data association and to enable a novel death model, all of which contribute to a more accurate and precise belief of the local scene. Simulation and experimental results are presented from a common autonomous driving scenario.

ICRA Conference 2012 Conference Paper

Execution and analysis of high-level tasks with dynamic obstacle anticipation

Benjamin Johnson 0002
Frank Havlak
Mark E. Campbell
Hadas Kress-Gazit

This paper uniquely embeds high-level robot controllers with sensor data obtained from abstracting probabilistic anticipation of the behavior of dynamic obstacles. An example problem of an autonomous vehicle operating in an urban environment, in the presence of other vehicles and pedestrians, is used as motivation. The correct-by-construction controller is automatically synthesized from a set of high-level tasks, specified as temporal logic formulas. The anticipated behavior of other vehicles is abstracted to a set of propositions describing the safety of road segments at intersections, and used as the output of high-level sensors for the controller. Such an input to the controller is inherently probabilistic, and this paper investigates the types of probabilistic guarantees that can be made about the system using both formal and statistical analysis.

IROS Conference 2012 Conference Paper

Iterative smoothing approach using Gaussian mixture models for nonlinear estimation

Daniel J. Lee
Mark E. Campbell

An iterative smoothing algorithm is developed using Gaussian mixture models in order to tackle challenging nonlinear estimation problems. Gaussian mixture models naturally capture nonlinear and non-Gaussian systems, while smoothing algorithms provide ability to update using measurements obtained in the past. A tree structure and Gaussian distribution splitting method are proposed to mitigate nonlinearity effects and complexities. Two methods, Children Collapsing and Parent Splitting, are developed to utilize sigma-points smoother for Gaussian mixture model. An indoor localization problem is used to explore and validate the approach. Performance of these new methods is compared to a baseline sigma-points smoother, in both simulation and experiment, and shows much improvement in overall error compared to the truth.

ICRA Conference 2011 Conference Paper

A robust qualitative planner for mobile robot navigation using human-provided maps

Danelle C. Shah
Mark E. Campbell

A novel method for controlling a mobile robot using qualitative inputs in the context of an approximate map, such as one sketched by a human, is presented. By defining a desired trajectory with respect to observable landmarks, human operators can send semi-autonomous robots into areas for which a truth map is not available. Waypoint planning is formulated as a quadratic optimization problem, resulting in robot trajectories in the true environment that are qualitatively similar to those provided by the human. The algorithm is implemented both in simulation and on a mobile robot platform in several different environments. A sensitivity analysis is performed, illustrating how the method is robust to uncertainties, even large sketch distortions, and allows the robot to adapt and re-plan according to its most current perception of the world.

IROS Conference 2011 Conference Paper

Clustering obstacle predictions to improve contingency planning for autonomous road vehicles in congested environments

Jason Hardy
Mark E. Campbell

A hierarchical trajectory clustering algorithm is presented with the goal of clustering a set of mutually exclusive obstacle trajectory predictions for use in a contingency based path planner for an autonomous road vehicle. This clustering algorithm improves the computational scaling of the contingency planner by limiting the total number of required contingency paths while preserving the performance advantages of exhaustive contingency planning. This algorithm seeks to maximize dissimilarity between trajectory clusters with regard to their potential effect on a robot's future path. Simulation results show that the clustering algorithm allows a robot to maintain many of the benefits of contingency planning while requiring fewer contingency paths.

IROS Conference 2010 Conference Paper

A robust sketch interface for natural robot control

Danelle C. Shah
Joseph Schneider
Mark E. Campbell

A fully probabilistic command interface for controlling robots using multi-stroke sketch commands is presented. Drawing from prior work in handwriting recognition, sketches are modeled as a variable duration hidden Markov model, where the distributions on the states and transitions are learned from training data. A forward search algorithm on the gesture, stroke, and stroke transition observations is used to find the most likely sketch, which is displayed to the user for confirmation. In cases where the most likely sketch is incorrect, the user can reject it, prompting the next most likely sketch to be displayed. Upon confirmation from the user, the robot executes the desired behaviors. A prototype sketch interface was implemented using a pen tablet; two sets of search-and-identify experiments were conducted using a single robot in an indoor environment to test the usability of the proposed framework. Even novice users were able to successfully complete the missions, including those on whom the algorithm was not trained. User surveys indicate that operators generally found the interface to be natural and easy to use.

IROS Conference 2010 Conference Paper

Contingency planning over probabilistic hybrid obstacle predictions for autonomous road vehicles

Jason Hardy
Mark E. Campbell

This paper presents a novel optimization based path planner that can simultaneously plan multiple contingency paths to account for the uncertain actions of dynamic obstacles. This planner addresses the particular problem of collision avoidance for autonomous road vehicles which are required to safely interact with other vehicles with unknown intentions. The presented path planner utilizes an efficient spline based trajectory representation and fast but accurate collision probability approximations to enable the simultaneous optimization of multiple contingency paths.

IROS Conference 2010 Conference Paper

Segmentation of dense range information in complex urban scenes

Jonathan R. Schoenberg
Aaron Nathan
Mark E. Campbell

In this paper, an algorithm to segment 3D points in dense range maps generated from the fusion of a single optical camera and a multiple emitter/detector laser range finder is presented. The camera image and laser range data are fused using a Markov Random Field to estimate a 3D point corresponding to each image pixel. The textured 3D dense point cloud is segmented based on evidence of a boundary between regions of the textured point cloud. Clusters are discriminated based on Euclidean distance, pixel intensity and estimated surface normal using a fast, deterministic and near linear time segmentation algorithm. The algorithm is demonstrated on data collected with the Cornell University DARPA Urban Challenge vehicle. Performance of the proposed dense segmentation routine is evaluated in a complex urban environment and compared to segmentation of the sparse point cloud. Results demonstrate the effectiveness of the dense segmentation algorithm to avoid over-segmentation better than incorporating color and surface normal data in the sparse point cloud.

ICRA Conference 2010 Conference Paper

Variational Bayesian data fusion of multi-class discrete observations with applications to cooperative human-robot estimation

Nisar R. Ahmed
Mark E. Campbell

A new method is presented for fusing conventional continuous sensor observations with discrete multi-categorical state-dependent information, which can be furnished by humans in many cooperative human-robot interaction problems. The hybrid likelihood function for mapping between continuous hidden states and categorical observations are specified via softmax models. Although softmax models avoid discretization of continuous states, they are challenging to implement for real-time data fusion since they are not analytically integrable. An approximation based on variational Bayesian (VB) methods is presented here to obtain fast closed-form Gaussian solutions to the desired posteriors in cases where the hidden continuous states have Gaussian pdfs. A joint human-robot target localization example illustrates the properties and utility of the VB hybrid fusion strategy, which also applies more generally to inference in hybrid Bayesian networks and mixture models.

ICRA Conference 2009 Conference Paper

Localization with multi-modal vision measurements in limited GPS environments using Gaussian Sum Filters

Jonathan R. Schoenberg
Mark E. Campbell
Isaac Miller

A Gaussian Sum Filter (GSF) with component extended Kalman filters (EKF) is proposed as an approach to localize an autonomous vehicle in an urban environment with limited GPS availability. The GSF uses vehicle relative vision-based measurements of known map features coupled with inertial navigation solutions to accomplish localization in the absence of GPS. The vision-based measurements are shown to have multi-modal measurement likelihood functions that are well represented as a weighted sum of Gaussian densities and the GSF is ideally suited to accomplish recursive Bayesian state estimation for this problem. A sequential merging technique is used for Gaussian mixture condensation in the posterior density approximation after fusing multi-modal measurements in the GSF to maintain mixture size over time. The representation of the posterior density with the GSF is compared over a common dataset against a benchmark particle filter solution. The Expectation-Maximization (EM) algorithm is used offline to determine the representational efficiency of the particle filter in terms of an effective number of Gaussian densities. The GSF with vision-based vehicle relative measurements is shown to remain converged using 37 minutes of recorded data from the Cornell University DARPA Urban Challenge (DUC) autonomous vehicle in an urban environment that includes a 32 minute GPS blackout.

ICRA Conference 2009 Conference Paper

Probabilistic estimation of Multi-Level terrain maps

César Rivadeneyra
Isaac Miller
Jonathan R. Schoenberg
Mark E. Campbell

Recent research has shown that robots can model their world with Multi-Level (ML) surface maps, which utilize dasiapatchespsila in a 2D grid space to represent various environment elevations within a given grid cell. Though these maps are able to produce 3D models of the environment while exploiting the computational feasibility of single elevation maps, they do not take into account in-plane uncertainty when matching measurements to grid cells or when grouping those measurements into dasiapatches. psila To respond to these drawbacks, this paper proposes to extend these ML surface maps into Probabilistic Multi-Level (PML) surface maps, which uses formal probability theory to incorporate estimation and modeling errors due to uncertainty. Measurements are probabilistically associated to cells near the nominal location, and are categorized through hypothesis testing into dasiapatchespsila via classification methods that incorporate uncertainty. Experimental results comparing the performances of the PML and ML surface mapping algorithms on representative objects found in both indoor and outdoor environments show that the PML algorithm outperforms the ML algorithm in most cases including in the presence of noisy and sparse measurements. The experimental results support the claim that the PML algorithm produces more densely populated, conservative representations of its environment with fewer measurements than the ML algorithm.

ICRA Conference 2008 Conference Paper

Particle filtering for map-aided localization in sparse GPS environments

Isaac Miller
Mark E. Campbell

This study presents the PosteriorPose algorithm, a Bayesian particle filtering approach for augmenting GPS and inertial navigation solutions with vision-based measurements of nearby lanes and stoplines referenced against a known map of environmental features. These relative measurements are shown to improve the quality of the navigation solution when GPS is available, and they are shown to keep the navigation solution converged in extended GPS blackouts. Measurements are incorporated with careful hypothesis testing and error modeling to account for non-Gaussian errors committed by vision-based detection algorithms. The PosteriorPose algorithm is implemented and validated in real-time on Cornell University’s 2007 DARPA Urban Challenge entry; experimental data is presented showing the algorithm outperforming a tightly-coupled GPS / Inertial Navigation solution both in full GPS coverage and in an extended GPS blackout.

IROS Conference 2008 Conference Paper

Scalable Bayesian human-robot cooperation in mobile sensor networks

Frédéric Bourgault
Aakash Chokshi
John Wang
Danelle C. Shah
Jonathan R. Schoenberg
Ramnath Iyer
Franco Cedano
Mark E. Campbell

In this paper, scalable collaborative human-robot systems for information gathering applications are approached as a decentralized Bayesian sensor network problem. Human-computer augmented nodes and autonomous mobile sensor platforms are collaborating on a peer-to-peer basis by sharing information via wireless communication network. For each node, a computer (onboard the platform or carried by the human) implements both a decentralized Bayesian data fusion algorithm and a decentralized Bayesian control negotiation algorithm. The individual node controllers iteratively negotiate anonymously with each other in the information space to find cooperative search plans based on both observed and predicted information that explicitly consider the platforms (humans and robots) motion models, their sensors detection functions, as well as the target arbitrary motion model. The results of a collaborative multi-target search experiment conducted with a team of four autonomous mobile sensor platforms and five humans carrying small portable computers with wireless communication are presented to demonstrate the efficiency of the approach.

ICRA Conference 2007 Conference Paper

Rao-Blackwellized Particle Filtering for Mapping Dynamic Environments

Isaac Miller
Mark E. Campbell

A general method for mapping dynamic environments using a Rao-Blackwellized particle filter is presented. The algorithm rigorously addresses both data association and target tracking in a single unified estimator. The algorithm relies on a Bayesian factorization to separate the posterior into: 1) a data association problem solved via particle filter; and 2) a tracking problem with known data associations solved by Kalman filters developed specifically for the ground robot environment. The algorithm is demonstrated in simulation and validated in the real world with laser range data, showing its practical applicability in simultaneously resolving data association ambiguities and tracking moving objects.

ICRA Conference 2007 Conference Paper

Towards Probabilistic Operator-Multiple Robot Decision Models

Mark E. Campbell
Frédéric Bourgault
Scott Galster
David Schneider 0003

Coupled operator-multiple vehicle systems are modelled in a unified framework using probabilistic graphs to yield a methodology for analyzing semi-autonomous systems. The framework uses conditional probabilistic dependencies between all elements, leading to a Bayesian network (BN) with probabilistic evaluation capability. Vehicle attitude/navigation states and target/classification states can be evaluated using nonlinear estimators such as the EKF, multiple model filter, information filter, or other approaches. Discrete operator decisions are being modeled as Bayesian network blocks, with conditional dependencies on the vehicle and tracking estimators. Initial decision models use combinations of softmax and discrete probability distributions.

IROS Conference 2006 Conference Paper

Probability Map Building Algorithms Design for an Unknown Dynamic Environment

Yongchun Fang
Mark E. Campbell
Bojun Ma

In this paper, we consider the problem of building a probability map for an unknown hostile environment by utilizing a team of UAVs. Specifically, we first present a centralized map building scheme for the Boeing open experimental platform (OEP) environment, the strategy is then modified into a decentralized map building algorithm to increase the robustness of the system. Some simulation results are provided to demonstrate the validity of the proposed algorithms