Author name cluster

Camillo J. Taylor

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

51 papers

1 author row

ICRA Conference 2025 Conference Paper

EvMAPPER: High-Altitude Orthomapping with Event Cameras

Fernando Cladera
Kenneth Chaney
M. Ani Hsieh
Camillo J. Taylor
Vijay Kumar 0001

Traditionally, unmanned aerial vehicles (UAVs) rely on CMOS-based cameras to collect images about the world below. One of the most successful applications of UAVs is to generate orthomosaics or orthomaps, in which a series of images are integrated to develop a larger map. However, using CMOS-based cameras with global or rolling shutters means that orthomaps are vulnerable to challenging light conditions, motion blur, and high-speed motion of independently moving objects (IMOs) under the camera. Event cameras are less sensitive to these issues, as their pixels trigger asynchronously on brightness changes. This work introduces the first orthomosaic approach using event cameras. We focus on addressing high-dynamic range and low-light problems in orthomosaics. In contrast to existing methods relying only on CMOS cameras, our approach enables map generation even in challenging light conditions, including direct sunlight and after sunset. The source code for EvMAPPER, the high-altitude hardware, and the dataset collected in this paper are available open source 1 1 https://evmapper.fcladera.com.

ICRA Conference 2024 Conference Paper

Enabling Large-scale Heterogeneous Collaboration with Opportunistic Communications

Fernando Cladera
Zachary Ravichandran
Ian D. Miller
M. Ani Hsieh
Camillo J. Taylor
Vijay Kumar 0001

Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-robot collaboration that enables large-scale exploration in the absence of continuous communications. MOCHA is based on a gossip communication protocol that allows robots to interact opportunistically whenever communication links are available, propagating information on a peer-to-peer basis. We demonstrate the performance of MOCHA through real-world experiments with commercial-off-the-shelf (COTS) communication hardware. We further explore the system’s scalability in simulation, evaluating the performance of our approach as the number of robots increases and communication ranges vary. Finally, we demonstrate how MOCHA can be tightly integrated with the planning stack of autonomous robots. We show a communication-aware planning algorithm for a high-altitude aerial robot executing a collaborative task while maximizing the amount of information shared with ground robots. The source code for MOCHA and the high-altitude UAV planning system is available open source 1.

ICRA Conference 2023 Conference Paper

Active Metric-Semantic Mapping by Multiple Aerial Robots

Xu Liu 0007
Ankit Prabhu
Fernando Cladera
Ian D. Miller
Lifeng Zhou 0001
Camillo J. Taylor
Vijay Kumar 0001

Traditional approaches for active mapping focus on building geometric maps. For most real-world applications, however, actionable information is related to semantically meaningful objects in the environment. We propose an approach to the active metric-semantic mapping problem that enables multiple heterogeneous robots to collaboratively build a map of the environment. The robots actively explore to minimize the uncertainties in both semantic (object classification) and geometric (object modeling) information. We represent the environment using informative but sparse object models, each consisting of a basic shape and a semantic class label, and characterize uncertainties empirically using a large amount of real-world data. Given a prior map, we use this model to select actions for each robot to minimize uncertainties. The performance of our algorithm is demonstrated through multi-robot experiments in diverse real-world environments. The proposed framework is applicable to a wide range of real-world problems, such as precision agriculture, infrastructure inspection, and asset mapping in factories.

IROS Conference 2022 Conference Paper

DSOL: A Fast Direct Sparse Odometry Scheme

Chao Qu
Shreyas S. Shivakumar
Ian D. Miller
Camillo J. Taylor

In this paper, we describe Direct Sparse Odometry Lite (DSOL), an improved version of Direct Sparse Odometry (DSO) [1]. We propose several algorithmic and implementation enhancements which speed up computation by a significant factor (on average 5x) even on resource-constrained platforms. The increase in speed allows us to process images at higher frame rates, which in turn provides better results on rapid motions. Our open-source implementation is available at https://github.com/versatran01/dso1.

ICRA Conference 2022 Conference Paper

Fast Footstep Planning on Uneven Terrain Using Deep Sequential Models

Hersh Sanghvi
Camillo J. Taylor

One of the fundamental challenges in realizing the potential of legged robots is generating plans to traverse challenging terrains. Control actions must be carefully selected so the robot will not crash or slip. The high dimensionality of the joint space makes directly planning low-level actions from onboard perception difficult, and control stacks that do not consider the low-level mechanisms of the robot in planning are ill-suited to handle fine-grained obstacles. One method for dealing with this is selecting footstep locations based on terrain characteristics. However, incorporating robot dynamics into footstep planning requires significant computation, much more than in the quasi-static case. In this work, we present an LSTM-based planning framework that learns probability distributions over likely footstep locations using both terrain lookahead and the robot's dynamics, and leverages the LSTM's sequential nature to find footsteps in linear time. Our framework can also be used as a module to speed up sampling-based planners. We validate our approach on a simulated one-legged hopper over a variety of uneven terrains.

ICRA Conference 2022 Conference Paper

LLOL: Low-Latency Odometry for Spinning Lidars

Chao Qu
Shreyas S. Shivakumar
Wenxin Liu 0002
Camillo J. Taylor

In this paper, we present a low-latency odometry system designed for spinning lidars. Many existing lidar odometry methods wait for an entire sweep from the lidar before processing the data. This introduces a large delay between the first laser firing and its pose estimate. To reduce this latency, we treat the spinning lidar as a streaming sensor and process packets as they arrive. This effectively distributes expensive operations across time, resulting in a very fast and lightweight system with a much higher throughput and lower latency. Our open source implementation is available at https://github.com/versatran01/llol.

ICRA Conference 2021 Conference Paper

UPSLAM: Union of Panoramas SLAM

Anthony Cowley
Ian D. Miller
Camillo J. Taylor

We present an empirical investigation of a new mapping system based on a graph of panoramic depth images. Panoramic images eﬃciently capture range measurements taken by a spinning lidar sensor, recording fine detail on the order of a few centimeters within maps of expansive scope on the order of tens of millions of cubic meters. The flexibility of the system is demonstrated by running the same mapping software against data collected by hand-carrying a paired lidar and IMU around a laboratory space at walking pace, moving them outdoors through a campus environment at running pace, driving the sensors on a small wheeled vehicle on- and off-road, flying the sensors through a forest, carrying them on the back of a legged robot navigating an underground coal mine, and mounting them on the roof of a car driven on public roads. The full 3D maps are built online with a median update time of less than ten milliseconds on an embedded NVIDIA Jetson AGX Xavier system.

ICRA Conference 2020 Conference Paper

PST900: RGB-Thermal Calibration, Dataset and Segmentation Network

Shreyas S. Shivakumar
Neil Rodrigues
Alex Zhou
Ian D. Miller
Vijay Kumar 0001
Camillo J. Taylor

In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean Challenge. Lastly, we propose a CNN architecture for fast semantic segmentation that combines both RGB and Thermal imagery in a way that leverages RGB imagery independently. We compare our method against the state-of-the-art and show that our method outperforms them in our dataset.

ICRA Conference 2020 Conference Paper

The Tiercel: A novel autonomous micro aerial vehicle that can map the environment by flying into obstacles

Yash Mulgaonkar
Wenxin Liu 0002
Dinesh Thakur
Kostas Daniilidis
Camillo J. Taylor
Vijay Kumar 0001

Autonomous flight through unknown environments in the presence of obstacles is a challenging problem for micro aerial vehicles (MAVs). A majority of the current state-of-art research assumes obstacles as opaque objects that can be easily sensed by optical sensors such as cameras or LiDARs. However in indoor environments with glass walls and windows, or scenarios with smoke and dust, robots (even birds) have a difficult time navigating through the unknown space. In this paper, we present the design of a new class of micro aerial vehicles that achieves autonomous navigation and are robust to collisions. In particular, we present the Tiercel MAV: a small, agile, light weight and collision-resilient robot powered by a cellphone grade CPU. Our design exploits contact to infer the presence of transparent or reflective obstacles like glass walls, integrating touch with visual perception for SLAM. The Tiercel is able to localize using visual-inertial odometry (VIO) running on board the robot with a single downward facing fisheye camera and an IMU. We show how our collision detector design and experimental set up enable us to characterize the impact of collisions on VIO. We further develop a planning strategy to enable the Tiercel to fly autonomously in an unknown space, sustaining collisions and creating a 2D map of the environment. Finally we demonstrate a swarm of three autonomous Tiercel robots safely navigating and colliding through an obstacle field to reach their objectives.

ICRA Conference 2020 Conference Paper

Vision-based Multi-MAV Localization with Anonymous Relative Measurements Using Coupled Probabilistic Data Association Filter

Ty Nguyen
Kartik Mohta
Camillo J. Taylor
Vijay Kumar 0001

We address the localization of robots in a multi-MAV system where external infrastructure like GPS or motion capture systems may not be available. Our approach lends itself to implementation on platforms with several constraints on size, weight, and power (SWaP). Particularly, our framework fuses the onboard VIO with the anonymous, visual-based robot-to-robot detection to estimate all robot poses in one common frame, addressing three main challenges: 1) the initial configuration of the robot team is unknown, 2) the data association between each vision-based detection and robot targets is unknown, and 3) the vision-based detection yields false negatives, false positives, inaccurate, and provides noisy bearing, distance measurements of other robots. Our approach extends the Coupled Probabilistic Data Association Filter [1] to cope with nonlinear measurements. We demonstrate the superior performance of our approach over a simple VIO-based method in a simulation with the measurement models statistically modeled using the real experimental data. We also show how onboard sensing, estimation, and control can be used for formation flight.

ICRA Conference 2019 Conference Paper

Evaluating the Effectiveness of Perspective Aware Planning with Panoramas

Daniel Mox
Anthony Cowley
M. Ani Hsieh
Camillo J. Taylor

In this work, we present an information based exploration strategy tailored for the generation of high resolution 3D maps. We employ RGBD panoramas because they have been shown to provide memory efficient high quality representations of space. Robots explore the environment by selecting locations with maximal Cauchy-Schwarz Quadratic Mutual Information (CSQMI) computed on an angle enhanced occupancy grid to collect these RGBD panoramas. By employing the angle enhanced occupancy grid, the resulting exploration strategy emphasizes perspective in addition to binary coverage. Furthermore, the goal selection strategy is improved by using image morphology to reduce the search space over which CSQMI is computed. We present experimental results demonstrating the improved performance in perception related tasks by capturing panoramas using this approach, near frontier exploration, and a control of logging images at regular intervals while teleoperating the robot through the workspace. Collect imagery was passed through an object detection library with our perspective aware approach yielding a greater number of successful detections compared to near frontier exploration.

ICRA Conference 2019 Conference Paper

Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements

Shreyas S. Shivakumar
Kartik Mohta
Bernd Pfrommer
Vijay Kumar 0001
Camillo J. Taylor

We present an approach to depth estimation that fuses information from a stereo pair with sparse range measurements derived from a LIDAR sensor or a range camera. The goal of this work is to exploit the complementary strengths of the two sensor modalities, the accurate but sparse range measurements and the ambiguous but dense stereo information. These two sources are effectively and efficiently fused by combining ideas from anisotropic diffusion and semi-global matching. We evaluate our approach on the KITTI 2015 and Middlebury 2014 datasets, using randomly sampled ground truth range measurements as our sparse depth input. We achieve significant performance improvements with a small fraction of range measurements on both datasets. We also provide qualitative results from our platform using the PMDTec Monstar sensor. Our entire pipeline runs on an NVIDIA TX-2 platform at 5Hz on 1280×1024 stereo images with 128 disparity levels.

ICRA Conference 2019 Conference Paper

The Open Vision Computer: An Integrated Sensing and Compute System for Mobile Robots

Morgan Quigley
Kartik Mohta
Shreyas S. Shivakumar
Michael Watterson
Yash Mulgaonkar
Mikael Arguedas
Ke Sun 0008
Sikang Liu 0002

In this paper we describe the Open Vision Computer (OVC) which was designed to support high speed, vision guided autonomous drone flight. In particular our aim was to develop a system that would be suitable for relatively small-scale flying platforms where size, weight, power consumption and computational performance were all important considerations. This manuscript describes the primary features of our OVC system and explains how they are used to support fully autonomous indoor and outdoor exploration and navigation operations on our Falcon 250 quadrotor platform.

ICRA Conference 2018 Conference Paper

Experiments in Fast, Autonomous, GPS-Denied Quadrotor Flight

Kartik Mohta
Ke Sun 0008
Sikang Liu 0002
Michael Watterson
Bernd Pfrommer
James Svacha
Yash Mulgaonkar
Camillo J. Taylor

High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to fly at speeds of more than 18 m/s. We provide an overview of our system and details about the specific component technologies that enable the high speed navigation capability of our platform. We demonstrate the robustness of our system through high speed autonomous flights and navigation through a variety of obstacle rich environments.

IROS Conference 2018 Conference Paper

Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion

Xu Liu 0007
Steven W. Chen
Shreyas Aditya
Nivedha Sivakumar
Sandeep Dcunha
Chao Qu
Camillo J. Taylor
Jnaneshwar Das

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.

IROS Conference 2016 Conference Paper

Recovering relative orientation and scale from visual odometry and ranging radio measurements

Armon Shariati
Kartik Mohta
Camillo J. Taylor

In this paper we propose a new approach to recovering the relative position and orientation of a pair of platforms moving on the plane by fusing the kinds of estimates provided by visual odometry systems with distance measurements obtained from ranging radio systems. By combining these two complementary sources of information we are able to provide estimates that could not be obtained using either source separately. We show that the localization problem can be phrased as an optimization problem where we are interested in maximizing a convex function subject to a set of convex constraints and we propose an efficient and effective solution scheme that leverages the structure of the problem. Experimental results are provided to demonstrate that the proposed method performs creditably on actual robotic platforms.

IROS Conference 2016 Conference Paper

Towards fully autonomous visual inspection of dark featureless dam penstocks using MAVs

Tolga Özaslan
Kartik Mohta
James Keller 0002
Yash Mulgaonkar
Camillo J. Taylor
Vijay Kumar 0001
Jennifer M. Wozencraft
Thomas Hood

In the last decade, multi-rotor Micro Aerial Vehicles (MAVs) have attracted great attention from robotics researchers. Offering affordable agility and maneuverability, multi-rotor aircrafts have become the most commonly used platforms for robotics applications. Amongst the most promising applications are inspection of power-lines, cell-towers, large and constrained infrastructures and precision agriculture. While GPS offers an easy solution for outdoor autonomy, using on-board sensors is the only solution for autonomy in constrained indoor environments. In this paper, we present our results on autonomous inspection of completely dark, featureless, symmetric dam penstocks using cameras and range sensors. We use a hex-rotor platform equipped with an IMU, four cameras and two lidars. One of the cameras tracks features on the walls using the on-board illumination to estimate the position along the tunnel axis unobservable to range sensors while all of the cameras are used for panoramic image construction. The two lidars estimate the remaining degrees of freedom (DOF). Outputs of the two estimators are fused using an Unscented Kalman Filter (UKF). A moderately trained operator defines waypoints using the Remote Control (RC). We demonstrate our results from Carters Dam, GA and Glen Canyon Dam, AZ which include panoramic images for cracks and rusty spot detection and 6-DOF estimation results with ground truth comparisons. To our knowledge ours is the only study that can autonomously inspect environments with no geometric cues and poor to no external illumination using MAVs.

IROS Conference 2015 Conference Paper

Mapping with depth panoramas

Camillo J. Taylor
Anthony Cowley
Rafe Kettler
Kai Ninomiya
Mayank Gupta
Boyang Niu

This work demonstrates the use of depth panoramas in the construction of detailed 3D models of extended environments. The paper describes an approach to the acquisition of such panoramic images using a robotic platform that collects sequences of depth images with a commodity depth sensor. These sequences are stitched into panoramic images that efficiently capture scene information while reducing noise in the captured imagery. Scene structure is extracted from the panoramas and used to register a collection of panoramas to obtain coverage of extended areas. The presented approach maintains fine geometry detail in 3D reconstructions, uses a modest amount of memory, and enjoys excellent scaling characteristics across large environments.

IROS Conference 2014 Conference Paper

Network localization from relative bearing measurements

Ryan Kennedy
Camillo J. Taylor

We present an approach for 2D sensor network localization when only bearing measurements are available and no global coordinate frame is known. Our work builds off of the linear constraint given in Kennedy et al. (2012) for sets of nodes that form triangles. We extend that constraint to general networks and present methods for locally optimizing the resulting cost function. We also show how these methods can be used for 3D network localization when the vertical axis is known. The algorithms are evaluated on both synthetic and real datasets, and we also show how they can be applied to the “structure from motion” problem in the field of computer vision.

IROS Conference 2013 Conference Paper

Perception and motion planning for pick-and-place of dynamic objects

Anthony Cowley
Benjamin J. Cohen
William Marshall
Camillo J. Taylor
Maxim Likhachev

Mobile manipulators have brought a new level of flexibility to traditional automation tasks such as tabletop manipulation, but are not yet capable of the same speed and reliability as industrial automation. We present approaches to 3D perception and manipulator motion planning that enable a general purpose robotic platform to recognize and manipulate a variety of objects at a rate of one pick-and-place operation every 6. 7 s, and work with a conveyor belt carrying objects at a speed of 33 cm /s.

IROS Conference 2012 Conference Paper

Identifying maximal rigid components in bearing-based localization

Ryan Kennedy
Kostas Daniilidis
Oleg Naroditsky
Camillo J. Taylor

We present an approach for sensor network localization when provided with a set of angular constraints. This problem arises in camera networks when angles between nearby points can be measured but depth measurements are not readily available. We provide contributions for two different variations on this problem. First, when each node is aware of a global coordinate frame, we present a novel method for identifying the components of the problem that are rigidly constrained. Second, in the more difficult case where only relative angles are known, we propose a novel spectral solution that achieves a globally-optimal embedding under transitively-triangular constraints, which we show encompass a wide range of real-world conditions. We demonstrate the utility of our algorithm on both synthetic data and data from quadrotor robot formations.

ICRA Conference 2011 Conference Paper

Fast scene analysis using image and range data

Camillo J. Taylor
Anthony Cowley

This paper presents a scheme which takes as input a 3D point cloud and an associated color image and parses the scene into a collection of salient planar surfaces. The scheme makes use of a fast color segmentation scheme to divide the color image into coherent regions and the groupings suggested by this procedure are used to inform and accelerate a RANSAC based interpretation process. Results on real data sets are presented.

ICRA Conference 2011 Conference Paper

Rapid multi-robot exploration with topometric maps

Anthony Cowley
Camillo J. Taylor
Ben Southall

Multi-robot map building has advanced to the point where high quality occupancy grid data may be collected by multiple robots collaborating with only intermittent connectivity. However, the tasking of these agents to most efficiently build the map is a problem that has seen less attention. Unfamiliar, highly cluttered environments can confound exploration strategies that rely solely on occupancy grid frontier identification or even semantic classification methods keyed on geometric features. To reason about partial maps of novel, highly cluttered locations, hypotheses about significant structure in the disposition of free space may be used to guide exploration task assignment. A parsing of map data into places with semantic significance to the exploration task provides a foundation from which one may infer an efficient exploration strategy.

IROS Conference 2011 Conference Paper

Stream-oriented robotics programming: The design of roshask

Anthony Cowley
Camillo J. Taylor

The decomposition of robotics software into a collection of loosely coupled processes has become a core design principle of virtually every large robotics software engineering effort over the past decade. Recently, the ROS software platform from Willow Garage has gained significant traction due to its adoption of sound design principles and significant software library contributions from Willow Garage itself. This paper describes a binding from the Haskell programming language to basic ROS interfaces. The novelty of these bindings is that they allow for, and encourage, a higher level of abstraction in writing programs for robots that treat streams of values as first-class citizens. This approach makes the fusing, transforming, and filtering of streams fully generic and compositional while maintaining full compatibility with the existing ROS ecosystem.

IROS Conference 2011 Conference Paper

Towards language-based verification of robot behaviors

Anthony Cowley
Camillo J. Taylor

The management of finite resources is central to many robot behaviors. Some robotic systems must maintain invariants regarding the disposition of feet for balancing, others have grippers for manipulating their environments, while yet others must respect strict rules governing the usage of objects in the environment. Yet the specifics of such resource management responsibilities are almost universally locked behind opaque controllers whose lack of type information greatly impedes rigorous static analysis. We present an application of dependent type theory and linear logic for the static analysis of robot behavior programs that manage both robot and environment state, with a worked assembly task example. This approach offers static, formal guarantees with respect to safety requirements attached to primitive actions, as well as introspection of expected state at each step of a scripted sequence of actions allowing for the automatic generation of dynamic, sensor-based, runtime verification of successful execution.

ICRA Conference 2009 Conference Paper

Expansion segmentation for visual collision detection and estimation

Jeffrey Byrne
Camillo J. Taylor

Collision detection and estimation from a monocular visual sensor is an important enabling technology for safe navigation of small or micro air vehicles in near earth flight. In this paper, we introduce a new approach called expansion segmentation, which simultaneously detects “collision danger regions” of significant positive divergence in inertial aided video, and estimates maximum likelihood time to collision (TTC) in a correspondenceless framework within the danger regions. This approach was motivated from a literature review which showed that existing approaches make strong assumptions about scene structure or camera motion, or pose collision detection without determining obstacle boundaries, both of which limit the operational envelope of a deployable system. Expansion segmentation is based on a new formulation of 6-DOF inertial aided TTC estimation, and a new derivation of a first order TTC uncertainty model due to subpixel quantization error and epipolar geometry uncertainty. Proof of concept results are shown in a custom designed urban flight simulator and on operational flight data from a small air vehicle.

IROS Conference 2007 Conference Paper

A bounded uncertainty approach to cooperative localization using relative bearing constraints

Camillo J. Taylor
John R. Spletzer

This paper describes an approach to cooperative localization which finds its roots in robust estimation, employing an unknown-but-bounded error model for sensor measurements. In this framework, range and bearing measurements obtained by the robots are viewed as constraints which implicitly define a set of feasible solutions in the joint configuration space of the robot team. The scheme produces bounded uncertainty estimates for the relative configuration of the team by using convex optimization techniques to approximate the projection of this feasible set onto various subspaces of the configuration space. The scheme can also be used to localize distributed sensor nodes. An important advantage of the proposed approach is that it is able to produce bounded uncertainty estimates for the relative configuration of the robots even in the case where the relative orientations of the robots are completely unknown. This is an important practical advance since errors in relative orientation are often a major contributor to positioning uncertainty in multi-robot localization schemes.

IROS Conference 2007 Conference Paper

Orchestrating concurrency in robot swarms

Anthony Cowley
Camillo J. Taylor

A functional approach to programming robot swarms brings with it well-defined properties that allow for automated concurrency and distributed execution. Further, the particular expressiveness of a pure functional language with first-class closures captures so cleanly certain biologically-inspired behaviors that program specification often becomes compact enough to allow a programmer to visually inspect the program code for the entire swarm at once. This benefit comes in contrast to more piece-meal construction methods used to build-up robot software from discrete components. While such programming models capture the engineered structure of a robotic system, the dynamic, decentralized qualities sought after in robot swarms are well-matched by the idioms of functional concurrent programming.

IROS Conference 2007 Conference Paper

Robustness and self-repair in modular robots

Mark Yim
Babak Shirmohammadi
Jimmy Sastra
Michael Park
Mike Dugan
Camillo J. Taylor

This video shows an uncut sequence of a 15 module robot in a bi-pedal configuration subject to a variety of disturbances yet staying on task. As the robot walks, a large disturbance is introduced. A human kicks the robot in its midsection causing the robot to fall into three separate clusters of five modules. Each of these pieces diagnose their state sensing gravity and their connectedness. They each perform a self-righting maneuver, then begin to search for each other. This is a first demonstration towards the development of solving larger self-reassembly problems that include greater levels of randomization and entropy (more pieces more widely distributed.) The 2006 Robotics Science and Systems workshop on self-reconfigurable robots developed as one of the grand challenges for this area, the ability to survive and self-repair after an explosion. This work will push the technologies required for integrated sensing, localization, distributed control, under highly unstructured conditions.

IROS Conference 2007 Conference Paper

Towards robotic self-reassembly after explosion

Mark Yim
Babak Shirmohammadi
Jimmy Sastra
Michael Park
Mike Dugan
Camillo J. Taylor

This paper introduces a new challenge problem: designing robotic systems to recover after disassembly from high-energy events and a first implemented solution of a simplified problem. It uses vision-based localization for self- reassembly. The control architecture for the various states of the robot, from fully-assembled to the modes for sequential docking, are explained and inter-module communication details for the robotic system are described.

ICRA Conference 2006 Conference Paper

Opening the Dialog: Robotics and the Internet

Anthony Cowley
Hwa-Chow Oliver Hsu
Camillo J. Taylor

Inter-component communication has received considerable attention by robotic software architects as various frameworks and toolkits have matured. While the resulting software platforms have proven useful for abstracting hardware interfaces and the complex networking issues that are often present in robot deployment scenarios, they typically present users with arduous paths for integrating new hardware and software, while making no allowance for humans as peers on the network. We explore how mature standards and protocols born of the Internet community can be leveraged to empower software designed for robots and sensor networks. Here we present software approaches that facilitate the integration of new hardware devices, software platforms, and humans in ways that are natural and intuitive for each

ICRA Conference 2006 Conference Paper

Towards the Deployment of a Mobile Robot Network with End-to-end Performance Guarantees

M. Ani Hsieh
Anthony Cowley
Vijay Kumar 0001
Camillo J. Taylor

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we present an experimental study of strategies for maintaining end-to-end communication links for tasks such as surveillance and search and rescue where team connectivity is essential for providing situational awareness to a base station. We consider the differences between monitoring point-to-point signal strength versus data throughput and present experimental results with our multi-robot testbed in outdoor environments

ICRA Conference 2005 Conference Paper

Planning and Control of Mobile Robots in Image Space from Overhead Cameras

Rahul Rao
Vijay Kumar 0001
Camillo J. Taylor

In this work, we present a framework for the development of a planar mobile robot controller based on image plane feedback. We show that the design of such a motion controller can be accomplished in the image plane by making use of a subset of the parameters that relate the image plane to the ground plane, while still leveraging the simplifications offered by modeling the system as a differentially flat system. Our method relies on a waypoint-based trajectory generator, with all the waypoints specified in the image, as seen by an overhead observer. We present some results from simulation as well as from experiments that validate the ideas presented in this work and discuss some ideas for future work.

ICRA Conference 2004 Conference Paper

Constructing Radio Signal Strength Maps with Multiple Robots

M. Ani Hsieh
Vijay Kumar 0001
Camillo J. Taylor

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we investigate the construction of a map of radio signal strength that can be used to plan multirobot tasks and also serve as useful perceptual information. We show how nominal models of an urban environment, such as those obtained by aerial surveillance, can be used to generate strategies for exploration and present preliminary experimental results with our multi-robot testbed.

ICRA Conference 2004 Conference Paper

Distributed Sensor Databases for Multi-robot Teams

Anthony Cowley
Hwa-Chow Oliver Hsu
Camillo J. Taylor

We describe our implementation of a distributed sensor database that was designed to support the activities of teams of mobile robots as they explore an environment. Importantly, this approach effectively separates the process of acquiring sensor data from that of exploiting it. This allows us to develop applications where robots and human users can automatically discover and utilize sensor measurements acquired by other robots in the team. We also explain our approach to implementing distributed queries, an important capability that allows us to perform queries in a way that makes best use of the limited available communication bandwidth. Finally, we briefly describe how we have used this system to support situational awareness tasks.

ICRA Conference 2004 Conference Paper

Experiments in Multirobot Air-Ground Coordination

Luiz Chaimowicz
Ben Grocholsky
James Keller 0002
Vijay Kumar 0001
Camillo J. Taylor

This paper addresses the problem of coordinating aerial and ground vehicles in tasks that involve exploration, identification of targets and maintaining a connected communication network. We focus on the problem of localizing vehicles in urban environments where GPS signals are often unreliable or unavailable. We first describe our multi-robot testbed and the control software used to coordinate ground and aerial vehicles. We present the results of experiments in air-ground localization analyzing three complementary approaches to determining the positions of vehicles on the ground. We show that the coordination of aerial vehicles with ground vehicles is necessary to get accurate estimates of the state of the system.

IROS Conference 2003 Conference Paper

A bounded uncertainty approach to multi-robot localization

John R. Spletzer
Camillo J. Taylor

We offer a new approach to the multi-robot localization problem. Using an unknown-but-bounded model for sensor error, we are able to define convex polytopes in the configuration space of the robot team that represent the set of configurations consistent with all sensor measurements. Estimates for the uncertainty in various parameters of the team's configuration such as the absolute position of a single robot, or the relative positions of two or more nodes can be obtained by projecting this polytope onto appropriately chosen subspaces of the configuration space. We propose a novel approach to approximating these projections using linear programming techniques. The approach can handle both bearing and range measurements with a computational complexity scaling polynomially in the number of roots. Finally, the workload is readily distributed - requiring only the communication of sensor measurements between robots. We provide simulation results for this approach implemented on a multi-robot team.

IROS Conference 2003 Conference Paper

Human robot interaction and usability studies for a smart wheelchair

Sarangi P. Parikh
Rahul Rao
Sang-Hack Jung
Vijay Kumar 0001
Jim Ostrowski 0001
Camillo J. Taylor

We build on previous work, on the development of a computer controlled wheelchair equipped with a suite of sensors and a novel interface for human-robot interaction. In this paper, we present experimental results and usability studies for the wheelchair. The architecture for human-robot interaction is hierarchical, with the lowest level corresponding to trajectory control, the intermediate level being behavioral and the highest level involving the composition of behaviors and navigation. Our experimental results illustrate the benefits of a shared-control paradigm where the human operator selects the appropriate behavior(s) or goals while the software is responsible for executing behaviors and generating safe trajectories. Experiments with human users highlight advantages of augmentation in wheelchairs.

ICRA Conference 2003 Conference Paper

Quadrotor control using dual camera visual feedback

Erdinç Altug
Jim Ostrowski 0001
Camillo J. Taylor

In this paper, a vision-based stabilization and output tracking control method for a four-rotor helicopter has been proposed. A novel 2 camera method has been described for estimating the full 6 DOF pose of the helicopter. This two camera system is consisting of a pan-tilt ground camera and an onboard camera. The pose estimation algorithm is compared in simulation to other methods (such as four point method, and a stereo method) and is shown to be less sensitive to feature detection errors on the image plane. The proposed pose estimation algorithm and non-linear control techniques have been implemented on a remote controlled quadrotor helicopter.

IROS Conference 2003 Conference Paper

ROCI: a distributed framework for multi-robot perception and control

Luiz Chaimowicz
Anthony Cowley
Vito Sabella
Camillo J. Taylor

This paper presents ROCI, a framework for developing applications for multi-robot teams. In ROCI, each robot is considered a node, which contains several modules and may export different types of services and capabilities to other nodes. Each node runs a kernel that mediates the interactions of the robots in a team. This kernel keeps an updated database of all nodes and the functionalities that they export. Multi-robot applications can be built dynamically by connecting modules that may be running on different nodes over the network. As an example, we present an obstacle avoidance task implemented using our framework and also discuss the use of ROCI in a multi-robot scenario.

IROS Conference 2003 Conference Paper

Target tracking with distributed sensors: the focus of attention problem

Volkan Isler
John R. Spletzer
Sanjeev Khanna
Camillo J. Taylor

In this paper, we investigate data fusion techniques for target tracking using distributed sensors. Specifically, we are interested in how pairs of bearing or range sensors can be best assigned to targets in order to minimize the expected error in the estimates. We refer to this as the focus of attention (FOA) problem. In its general form, FOA is NP-hard and not well approximable. However, for specific geometries we obtain significant approximation results: a 2-approximation algorithm for stereo cameras on a line, a PTAS for when the cameras are equidistant, and a 1. 42 approximation for equally spaced range sensors on a circle. By reposing as a maximization problem - where the goal is to maximize the number of tracks with bounded error - we are able to leverage results from maximum set-packing to render the problem approximable. We demonstrate the results in simulation for a target tracking task, and for localizing a team of mobile agents in a sensor network. These results provide insights into sensor/target assignment strategies, as well as sensor placement in a distributed network.

IROS Conference 2003 Conference Paper

Visual servoing of a UGV from a UAV using differential flatness

Rahul Rao
Vijay Kumar 0001
Camillo J. Taylor

In this paper the problem of controlling the motion of a nonholonomic vehicle along a desired trajectory using observations from an overhead camera is considered. The control problem is formulated in the image plane. We show that the system in the image plane is differentially flat and use this property to generate effective control strategies using only visual feedback. Simulation results illustrate the methodology and show robustness to errors in the camera calibration parameters.

ICRA Conference 2002 Conference Paper

Human Robot Interaction: Application to Smart Wheelchairs

R. S. Rao
K. Conn
Sang-Hack Jung
Jay Katupitiya
Terry Kientz
Vijay Kumar 0001
Jim Ostrowski 0001
Sarangi Patel

Addresses the problem of human robot interaction with application to the design of assistive devices. We describe the design and development of a prototype of a smart wheelchair that can be commanded by a rider. Specifically, we focus on (a) the vision-based human interaction interface; (b) the suite of sensors on the chair, and (c) the software architecture and the control algorithms used to control the chair.

ICRA Conference 2002 Conference Paper

Sensor Based Door Navigation for a Nonholonomic Vehicle

Sarangi Patel
Sang-Hack Jung
Jim Ostrowski 0001
Rahul Rao
Camillo J. Taylor

This paper presents a sensor based algorithm for guiding a nonholonomic platform, such as a wheelchair, through a doorway. The controller uses information from a camera system and a laser range finder to perform image-based navigation. Simulations of the resultant switching controller are presented along with experimental results. A simple obstacle avoidance algorithm is also implemented on the experimental platform. Finally, we have considered the input of limited field-of-view constraints on this controller. All of these components together lead to a modal, image-based approach that will safely and robustly navigate a nonholonomic robot with sensor constraints through a doorway.

ICRA Conference 2002 Conference Paper

Sensor Planning and Control in a Dynamic Environment

John R. Spletzer
Camillo J. Taylor

This paper presents an approach to the problem of controlling the configuration of a team of mobile agents equipped with cameras so as to optimize the quality of the estimates derived from their measurements. The issue of optimizing the robots' configuration is particularly important in the context of teams equipped with vision sensors since most estimation schemes of interest will involve some form of triangulation. We provide a theoretical framework for tackling the sensor planning problem and a practical computational strategy, inspired by work on particle filtering, for implementing the approach. We extend our previous work by showing how modeled system dynamics and configuration space obstacles can be handled. These ideas have been demonstrated both in simulation and on actual robotic platforms. The results indicate that the framework is able to solve fairly difficult sensor planning problems online without requiring excessive amounts of computational resources.

IROS Conference 2001 Conference Paper

Cooperative localization and control for multi-robot manipulation

John R. Spletzer
Aveek K. Das
Rafael Fierro
Camillo J. Taylor
Vijay Kumar 0001
Jim Ostrowski 0001

We describe a framework for coordinating multiple robots in cooperative manipulation tasks in which vision is used for establishing relative position and orientation and maintaining formation. The two key contributions are a cooperative scheme for localizing the robots based on visual imagery that is more robust than decentralized localization, and a set of control algorithms that allow the robots to maintain a prescribed formation (shape and size). The ability to maintain a prescribed formation allows the robots to "trap" objects in their midst, and to "flow" the formation to a desired position. We derive the cooperative localization and control algorithms and present experimental results that illustrate the implementation and the performance of these algorithms.

ICRA Conference 2001 Conference Paper

Real-Time Vision-Based Control of a Nonholonomic Mobile Robot

Aveek K. Das
Rafael Fierro
Vijay Kumar 0001
Ben Southall
John R. Spletzer
Camillo J. Taylor

This paper considers the problem of vision-based control of a nonholonomic mobile robot. We describe the design and implementation of real-time estimation and control algorithms on a car-like robot platform using a single omni-directional camera as a sensor without explicit use of odometry. We provide experimental results for each of these vision-based control objects. The algorithms are packaged as control modes and can be combined hierarchically to perform higher level tasks involving multiple robots.

IROS Conference 2000 Conference Paper

A mode-based sensor fusion approach to robotic stair-climbing

Solomon Steplight
Geofrey Egnal
Sang-Hack Jung
Daniel B. Walker
Camillo J. Taylor
Jim Ostrowski 0001

We present a software architecture used to demonstrate the ability of a tank-like robot to climb multiple flights of stairs. The algorithm we use incorporates intelligent sensor fusion and a hierarchical modular structure. This structure simplifies the overall design, enables reuse, and promotes extensibility. The modular nature of the design also allows for a variety of user interface modes with different levels of user autonomy. The robustness of our algorithm allows us to ascend and descend stairs autonomously in many environments with different textures and configurations, all with minimal adjustment to the algorithm. Our approach develops an extensible sensor and task model that reduces the effect of sensor inaccuracies and geometric constraints, uses minimal information about the environment, and utilizes characteristic features in the environment for navigation.

ICRA Conference 2000 Conference Paper

Robust Vision-Based Pose Control

Camillo J. Taylor
Jim Ostrowski 0001

The problem of controlling the spatial position and orientation of a robotic platform based on the image data obtained from a video camera mounted on that platform is considered. More specifically, we propose control laws that will cause the robot to achieve and maintain a fixed position and orientation with respect to a set of feature points in the scene. We demonstrate analytically that the proposed control scheme is globally convergent even in the presence of large calibration errors in both the intrinsic parameters of the camera and in the extrinsic parameters which relate the frame of reference of the camera to the body frame of the robot platform which is being controlled. Furthermore, no a priori knowledge about the structure of the scene is assumed.

ICRA Conference 1998 Conference Paper

A Comparative Study of Vision-Based Lateral Control Strategies for Autonomous Highway Driving

Jana Kosecka
Robert Blasi
Camillo J. Taylor
Jitendra Malik

This paper will present the results of a comparative study of a set of vision-based control strategies that have been applied to the problem of steering an autonomous vehicle along a highway. The aim of this work has been to further our understanding of the characteristics of various control laws that could be applied to this problem with a view to making informed design decisions. The control strategies that we explored include a lead lag control law, a full-state linear controller and input-output linearizing control law. Each of these control strategies was implemented and tested on our experimental vehicle, a Honda Accord LX, both with and without a curvature feedforward component.

ICRA Conference 1992 Conference Paper

Structure and motion from line segments in multiple images

Camillo J. Taylor
David J. Kriegman

An approach for recovering the structure of a rigid scene composed of straight line segments and the pose of moving camera from multiple images under perspective projection is presented. Recovery is formulated as minimizing the total squared image distance between measured segments and the projection of the reconstructed infinite lines with respect to structural and motion parameters. An efficient algorithm for minimizing this nonlinear objective function is presented. The approach can be directly used for accurately locating correspondences in multieyed stereo, as well as for pose estimation from landmarks. Its effectiveness is demonstrated quantitatively through simulation and qualitatively with a real image sequence. >