Author name cluster

Vijay Kumar 0001

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

238 papers

1 author row

ICRA Conference 2025 Conference Paper

ADMM-MCBF-LCA: A Layered Control Architecture for Safe Real-Time Navigation

Anusha Srikanthan
Yifan Xue
Vijay Kumar 0001
Nikolai Matni
Nadia Figueroa

We consider the problem of safe real-time navigation of a robot in a dynamic environment with moving obstacles of arbitrary smooth geometries and input saturation constraints. We assume that the robot detects and models nearby obstacle boundaries with a short-range sensor and that this detection is error-free. This problem presents three main challenges: i) input constraints, ii) safety, and iii) real-time computation. To tackle all three challenges, we present a layered control architecture (LCA) consisting of an offline path library generation layer, and an online path selection and safety layer. To overcome the limitations of reactive methods, our offline path library consists of feasible controllers, feedback gains, and reference trajectories. To handle computational burden and safety, we solve online path selection and generate safe inputs that run at 100 Hz. Through simulations on Gazebo and Fetch hardware in an indoor environment, we evaluate our approach against baselines that are layered, end - to-end, or reactive. Our experiments demonstrate that among all algorithms, only our proposed LCA is able to complete tasks such as reaching a goal, safely. When comparing metrics such as safety, input error, and success rate, we show that our approach generates safe and feasible inputs throughout the robot execution.

ICRA Conference 2025 Conference Paper

An Active Perception Game for Robust Information Gathering

Siming He
Yuezhan Tao
Igor Spasojevic
Vijay Kumar 0001
Pratik Chaudhari

Active perception approaches select future viewpoints by using some estimate of the information gain. An inaccurate estimate can be detrimental in critical situations, e. g. , locating a person in distress. However the true information gained can only be calculated post hoc, i. e. , after the observation is realized. We present an approach to estimate the discrepancy between the estimated information gain (which is the expectation over putative future observations while neglecting correlations among them) and the true information gain. The key idea is to analyze the mathematical relationship between active perception and the estimation error of the information gain in a gametheoretic setting. Using this, we develop an online estimation approach that achieves sub-linear regret (in the number of timesteps) for the estimation of the true information gain and reduces the sub-optimality of active perception systems. We demonstrate our approach 1 1 Code is available at https://github.com/grasp-lyd/active-perception-game. Proofs are available at https://arxiv.org/abs/2404.00769.for active perception using a comprehensive set of experiments on: (a) different types of environments, including a quadrotor in a photorealistic simulation, real-world robotic data, and real-world experiments with ground robots exploring indoor and outdoor scenes; (b) different types of robotic perception data; and (c) different map representations. On average, our approach reduces information gain estimation errors by 42%, increases the information gain by 7%, PSNR by 5%, and semantic accuracy (measured as the number of objects that are localized correctly) by 6%. In real-world experiments with a Jackal ground robot, our approach demonstrated complex trajectories to explore occluded regions.

ICRA Conference 2025 Conference Paper

Constrained Learning for Decentralized Multi-Objective Coverage Control

Juan Cerviño
Saurav Agarwal
Vijay Kumar 0001
Alejandro Ribeiro

The multi-objective coverage control problem requires a robot swarm to collaboratively provide sensor coverage to multiple heterogeneous importance density fields (IDFs) simultaneously. We pose this as an optimization problem with constraints and study two different formulations: (1) Fair coverage, where we minimize the maximum coverage cost for any field, promoting equitable resource distribution among all fields; and (2) Constrained coverage, where each field must be covered below a certain cost threshold, ensuring that critical areas receive adequate coverage according to predefined importance levels. We study the decentralized setting where robots have limited communication and local sensing capabilities, making the system more realistic, scalable, and robust. Given the complexity, we propose a novel decentralized constrained learning approach that combines primal-dual optimization with a Learnable Perception-Action-Communication (LPAC) neural network architecture. We show that the Lagrangian of the dual problem can be reformulated as a linear combination of the IDFs, enabling the LPAC policy to serve as a primal solver. We empirically demonstrate that the proposed method (i) significantly outperforms state-of-the-art decentralized controllers by 30% on average in terms of coverage cost, (ii) transfers well to larger environments with more robots, and (iii) is scalable in the number of IDFs and robots in the swarm.

ICRA Conference 2025 Conference Paper

EvMAPPER: High-Altitude Orthomapping with Event Cameras

Fernando Cladera
Kenneth Chaney
M. Ani Hsieh
Camillo J. Taylor
Vijay Kumar 0001

Traditionally, unmanned aerial vehicles (UAVs) rely on CMOS-based cameras to collect images about the world below. One of the most successful applications of UAVs is to generate orthomosaics or orthomaps, in which a series of images are integrated to develop a larger map. However, using CMOS-based cameras with global or rolling shutters means that orthomaps are vulnerable to challenging light conditions, motion blur, and high-speed motion of independently moving objects (IMOs) under the camera. Event cameras are less sensitive to these issues, as their pixels trigger asynchronously on brightness changes. This work introduces the first orthomosaic approach using event cameras. We focus on addressing high-dynamic range and low-light problems in orthomosaics. In contrast to existing methods relying only on CMOS cameras, our approach enables map generation even in challenging light conditions, including direct sunlight and after sunset. The source code for EvMAPPER, the high-altitude hardware, and the dataset collected in this paper are available open source 1 1 https://evmapper.fcladera.com.

ICRA Conference 2025 Conference Paper

Jailbreaking LLM-Controlled Robots

Alexander Robey
Zachary Ravichandran
Vijay Kumar 0001
Seyed Hamed Hassani
George J. Pappas

The recent introduction of large language models (LLMs) has revolutionized the field of robotics by enabling contextual reasoning and intuitive human-robot interaction in domains as varied as manipulation, locomotion, and self-driving vehicles. When viewed as a stand-alone technology, LLMs are known to be vulnerable to jailbreaking attacks, wherein mali-cious prompters elicit harmful text by bypassing LLM safety guardrails. To assess the risks of deploying LLMs in robotics, in this paper, we introduce ROBOPAIR, the first algorithm designed to jailbreak LLM-controlled robots. Unlike existing, textual attacks on LLM chatbots, Robopairelicits harmful physical actions from LLM-controlled robots, a phenomenon we experimentally demonstrate in three scenarios: (i) a white-box setting, wherein the attacker has full access to the NVID IA Dolphins self-driving LLM, (ii) a gray-box setting, wherein the attacker has partial access to a Clearpath Robotics Jackal UGV robot equipped with a GPT-40 planner, and (iii) a black-box setting, wherein the attacker has only query access to the GPT-3. 5-integrated Unitree Robotics Go2robot dog. In each scenario and across three new datasets of harmful robotic actions, we demonstrate that ROBOPAIR, as well as several static baselines, finds jailbreaks quickly and effectively, often achieving 100 % attack success rates. Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world. Indeed, our results on the U nitree G02represent the first successful jailbreak of a deployed commercial robotic system. Addressing this emerging vulnerability is critical for ensuring the safe deployment of LLMs in robotics. Additional media is available at: https://robopair.org.

ICRA Conference 2025 Conference Paper

Leveraging Symmetry to Accelerate Learning of Trajectory Tracking Controllers for Free-Flying Robotic Systems

Jake Welde
Nishanth Rao
Pratik Kunapuli
Dinesh Jayaraman
Vijay Kumar 0001

Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynamics and modest online compute budgets. However, the poor sample efficiency of RL and the challenges of reward design make training slow and sometimes unstable, especially for high-dimensional systems. In this work, we leverage the inherent Lie group symmetries of robotic systems with a floating base to mitigate these challenges when learning tracking controllers. We model a general tracking problem as a Markov decision process (MDP) that captures the evolution of both the physical and reference states. Next, we prove that symmetry in the underlying dynamics and running costs leads to an MDP homomorphism, a mapping that allows a policy trained on a lower-dimensional “quotient” MDP to be lifted to an optimal tracking controller for the original system. We compare this symmetry-informed approach to an unstructured baseline, using Proximal Policy Optimization (PPO) to learn tracking controllers for three systems: the Particle (a forced point mass), the Astrobee (a fully-actuated space robot), and the Quadrotor (an underactuated system). Results show that a symmetry-aware approach both accelerates training and reduces tracking error at convergence.

IROS Conference 2025 Conference Paper

Resilient Multi-Robot Target Tracking with Sensing and Communication Danger Zones

Peihan Li
Yuwei Wu 0005
Jiazhen Liu
Gaurav S. Sukhatme
Vijay Kumar 0001
Lifeng Zhou 0001

Multi-robot collaboration for target tracking in adversarial environments poses significant challenges, including system failures, dynamic priority shifts, and other unpredictable factors. These challenges become even more pronounced when the environment is unknown. In this paper, we propose a resilient coordination framework for multi-robot, multi-target tracking in environments with unknown sensing and communication danger zones. We consider scenarios where failures caused by these danger zones are probabilistic and temporary, allowing robots to escape from danger zones to minimize the risk of future failures. We formulate this problem as a nonlinear optimization with soft chance constraints, enabling real-time adjustments to robot behaviors based on varying types of dangers and failures. This approach dynamically balances target tracking performance and resilience, adapting to evolving sensing and communication conditions in real-time. To validate the effectiveness of the proposed method, we assess its performance across various tracking scenarios, benchmark it against methods without resilient adaptation and collaboration, and conduct several real-world experiments.

ICRA Conference 2025 Conference Paper

Safe Interval Motion Planning for Quadrotors in Dynamic Environments

Songhao Huang
Yuwei Wu 0005
Yuezhan Tao
Vijay Kumar 0001

Trajectory generation in dynamic environments presents a significant challenge for quadrotors, particularly due to the non-convexity in the spatial-temporal domain. Many existing methods either assume simplified static environments or struggle to produce optimal solutions in real-time. In this work, we propose an efficient safe interval motion planning framework for navigation in dynamic environments. A safe interval refers to a time window during which a specific configuration is safe. Our approach addresses trajectory generation through a two-stage process: a front-end graph search step followed by a back-end gradient-based optimization. We ensure completeness and optimality by constructing a dynamic connected visibility graph and incorporating low-order dynamic bounds within safe intervals and temporal corridors. To avoid local minima, we propose a Uniform Temporal Visibility Deformation (UTVD) for the complete evaluation of spatial-temporal topological equivalence. We represent trajectories with B-Spline curves and apply gradient-based optimization to navigate around static and moving obstacles within spatial-temporal corridors. Through simulation and real-world experiments, we show that our method can achieve a success rate of over $\mathbf{9 5 \%}$ in environments with different density levels, exceeding the performance of other approaches, demonstrating its potential for practical deployment in highly dynamic environments.

ICRA Conference 2025 Conference Paper

SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environments

Zachary Ravichandran
Varun Murali
Mariliza Tzes
George J. Pappas
Vijay Kumar 0001

As robots become increasingly capable, users will want to describe high-level missions and have robots infer the relevant details. Because pre-built maps are difficult to obtain in many realistic settings, accomplishing such missions will require the robot to map and plan online. While many semantic planning methods operate online, they are typically designed for well specified missions such as object search or exploration. Recently, Large Language Models (LLMs) have demonstrated powerful contextual reasoning abilities over a range of robotic tasks described in natural language. However, existing LLM-enabled planners typically do not consider online planning or complex missions; rather, relevant subtasks and semantics are provided by a pre-built map or a user. We address these limitations via SPINE, an online planner for missions with incomplete mission specifications provided in natural language. The planner uses an LLM to reason about subtasks implied by the mission specification and then realizes these subtasks in a receding horizon framework. Tasks are automatically validated for safety and refined online with new map observations. We evaluate SPINE in simulation and real-world settings with missions that require multiple steps of semantic reasoning and exploration in cluttered outdoor environments of over 20, 000m 2. Compared to baselines that use existing LLM-enabled planning approaches, our method is over twice as efficient in terms of time and distance, requires less user interactions, and does not require a full map. Additional resources are provided at https://zacravichandran.github.io/SPINE.

ICRA Conference 2025 Conference Paper

Towards Safe and Energy-Efficient Real-Time Motion Planning in Windy Urban Environments

Spencer Folk
John Melton
Benjamin W. L. Margolis
Mark Yim
Vijay Kumar 0001

Urban winds are a serious hazard for low-altitude autonomous aerial operations in urban airspaces. Previous methods for motion planning in urban winds require global knowledge of the obstacles and flow field and do not lend themselves to real-time application. In this paper, a planning and control framework is proposed for safe and energy-efficient navigation through urban flow fields that strictly relies on onboard sensing. The algorithm incorporates predictions of local wind flow fields into a receding horizon optimal controller, balancing energy consumption with obstacle avoidance on the fly to reach a goal destination. Simulation studies on a procedurally generated urban map with diverse wind conditions demonstrate that the energy-aware motion planner reduces energy consumption by as much as 30% and results in 32% fewer crashes on average compared to the wind-agnostic baseline. Comparisons to a global wind-aware planner indicate only minor trade-offs associated with planning on a local horizon.

ICRA Conference 2025 Conference Paper

Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance

Anish Bhattacharya
Nishanth Rao
Dhruv Parikh
Pratik Kunapuli
Yuwei Wu 0005
Yuezhan Tao
Nikolai Matni
Vijay Kumar 0001

We demonstrate the capabilities of an attentionbased end-to-end approach for high-speed vision-based quadrotor obstacle avoidance in dense, cluttered environments, with comparison to various state-of-the-art learning architectures. Quadrotor unmanned aerial vehicles (UAVs) have tremendous maneuverability when flown fast; however, as flight speed increases, traditional model-based approaches to navigation via independent perception, mapping, planning, and control modules breaks down due to increased sensor noise, compounding errors, and increased processing latency. Thus, learning-based, end-to-end vision-to-control networks have shown to have great potential for online control of these fast robots through cluttered environments. We train and compare convolutional, U-Net, and recurrent architectures against vision transformer (ViT) models for depth image-to-control in high-fidelity simulation, observing that ViT models are more effective than others as quadrotor speeds increase and in generalization to unseen environments, while the addition of recurrence further improves performance while reducing quadrotor energy cost across all tested flight speeds. We assess performance at speeds of up to 7m/s in simulation and hardware. To the best of our knowledge, this is the first work to utilize vision transformers for end-to-end vision-based quadrotor control.

ICRA Conference 2024 Conference Paper

Design and Evaluation of Motion Planners for Quadrotors in Environments with Varying Complexities

Yifei Simon Shao
Yuwei Wu 0005
Laura Jarin-Lipschitz
Pratik Chaudhari
Vijay Kumar 0001

Motion planning techniques for quadrotors have advanced significantly over the past decade. Most successful planners have two stages: a front-end that determines a path that incorporates geometric (or kinematic or input) constraints and specifies the homotopy class of the trajectory, and a back-end that optimizes this path to respect dynamics and input constraints. While there are many different choices for each stage, the eventual performance depends critically not only on these choices, but also on the environment. Given a new environment, it is difficult to decide a priori how one should design a motion planner. In this work, we develop (i) a procedure to construct parametrized environments, (ii) metrics that characterize the difficulty of motion planning in these environments, and (iii) an open-source software stack that can be used to combine a wide variety of two-stage planners seamlessly. We perform experiments in simulations and a real platform. We find, somewhat conveniently, that geometric front-ends are sufficient for environments with varying complexities if combined with dynamics-aware backends. The metrics we designed faithfully capture the planning difficulty in a given environment. All code is available at https://github.com/KumarRobotics/kr_mp_design.

ICRA Conference 2024 Conference Paper

Enabling Large-scale Heterogeneous Collaboration with Opportunistic Communications

Fernando Cladera
Zachary Ravichandran
Ian D. Miller
M. Ani Hsieh
Camillo J. Taylor
Vijay Kumar 0001

Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-robot collaboration that enables large-scale exploration in the absence of continuous communications. MOCHA is based on a gossip communication protocol that allows robots to interact opportunistically whenever communication links are available, propagating information on a peer-to-peer basis. We demonstrate the performance of MOCHA through real-world experiments with commercial-off-the-shelf (COTS) communication hardware. We further explore the system’s scalability in simulation, evaluating the performance of our approach as the number of robots increases and communication ranges vary. Finally, we demonstrate how MOCHA can be tightly integrated with the planning stack of autonomous robots. We show a communication-aware planning algorithm for a high-altitude aerial robot executing a collaborative task while maximizing the amount of information shared with ground robots. The source code for MOCHA and the high-altitude UAV planning system is available open source 1.

ICRA Conference 2024 Conference Paper

Learning to Explore Indoor Environments using Autonomous Micro Aerial Vehicles

Yuezhan Tao
Eran Iceland
Beiming Li
Elchanan Zwecher
Uri Heinemann
Avraham Cohen
Amir Avni
Oren Gal

In this paper, we address the challenge of exploring unknown indoor environments using autonomous aerial robots with Size Weight and Power (SWaP) constraints. The SWaP constraints induce limits on mission time requiring efficiency in exploration. We present a novel exploration framework that uses Deep Learning (DL) to predict the most likely indoor map given the previous observations, and Deep Reinforcement Learning (DRL) for exploration, designed to run on modern SWaP constraints neural processors. The DL-based map predictor provides a prediction of the occupancy of the unseen environment while the DRL-based planner determines the best navigation goals that can be safely reached to provide the most information. The two modules are tightly coupled and run onboard allowing the vehicle to safely map an unknown environment. Extensive experimental and simulation results show that our approach surpasses state-of-the-art methods by 50-60% in efficiency, which we measure by the fraction of the explored space as a function of the trajectory length.

ICRA Conference 2024 Conference Paper

Opportunistic Communication in Robot Teams

Daniel Mox
Kashish Garg
Alejandro Ribeiro
Vijay Kumar 0001

In this paper we present a new approach to Mobile Infrastructure on Demand (MID) where a dedicated team of robots creates and sustains a wireless network that satisfies the communication requirements of a different team of task-oriented robots seeking to coordinate their actions in the absence of existing communication infrastructure. Different from previous works, our approach forgoes heuristics for network performance such as algebraic-connectivity or network flow optimizations and instead positions communication support robots to directly maximize the probability of packet delivery by the underlying opportunistic routing protocol. Our system is task agnostic and practical to implement and operate on robots equipped with off-the-shelf WiFi radios. We demonstrate this through a set of experiments showing our MID system maintaining the delivery of critical mission data in a situational awareness setting and enabling foraging robots to effectively coordinate their actions during multi-robot exploration.

IROS Conference 2024 Conference Paper

TOPPQuad: Dynamically-Feasible Time-Optimal Path Parametrization for Quadrotors

Katherine Mao
Igor Spasojevic
M. Ani Hsieh
Vijay Kumar 0001

Planning time-optimal trajectories for quadrotors in cluttered environments is a challenging, non-convex problem. This paper addresses minimizing the traversal time of a given collision-free geometric path without violating actuation bounds of the vehicle. Previous approaches have either relied on convex relaxations that do not guarantee dynamic feasibility or have generated overly conservative time parametrizations. We propose TOPPQuad, a time-optimal path parameterization algorithm for quadrotors which explicitly incorporates quadrotor rigid body dynamics and constraints, such as bounds on inputs (including motor thrusts) and state of the vehicle (including the pose, linear and angular velocity and acceleration). We demonstrate the ability of the planner to generate faster trajectories that respect hardware constraints of the robot compared to planners with relaxed notions of dynamic feasibility in both simulation and hardware. We also demonstrate how TOPPQuad can be used to plan trajectories for quadrotors that utilize bidirectional motors. Overall, the proposed approach paves a way towards maximizing the efficacy of autonomous micro aerial vehicles while ensuring their safety.

IROS Conference 2024 Conference Paper

Trajectory Optimization with Global Yaw Parameterization for Field-of-View Constrained Autonomous Flight

Yuwei Wu 0005
Yuezhan Tao
Igor Spasojevic
Vijay Kumar 0001

Trajectory generation for quadrotors with limited field-of-view sensors has numerous applications such as aerial exploration, coverage, inspection, videography, and target tracking. Most previous works simplify the task of optimizing yaw trajectories by either aligning the heading of the robot with its velocity, or potentially restricting the feasible space of candidate trajectories by using a limited yaw domain to circumvent angular singularities. In this paper, we propose a novel global yaw parameterization method for trajectory optimization that allows a 360-degree yaw variation as demanded by the underlying algorithm. This approach effectively bypasses inherent singularities by including supplementary quadratic constraints and transforming the final decision variables into the desired state representation. This method significantly reduces the needed control effort, and improves optimization feasibility. Furthermore, we apply the method to several examples of different applications that require jointly optimizing over both the yaw and position trajectories. Ultimately, we present a comprehensive numerical analysis and evaluation of our proposed method in both simulation and real-world experiments.

ICRA Conference 2024 Conference Paper

TreeScope: An Agricultural Robotics Dataset for LiDAR-Based Mapping of Trees in Forests and Orchards

Derek Cheng
Fernando Cladera
Ankit Prabhu
Xu Liu 0007
Alan Zhu 0002
Patrick Corey Green
Reza Ehsani
Pratik Chaudhari

Data collection for forestry, timber, and agriculture relies on manual techniques which are labor-intensive and time-consuming. We seek to demonstrate that robotics offers improvements over these techniques and can accelerate agricultural research, beginning with semantic segmentation and diameter estimation of trees in forests and orchards. We present TreeScope v1. 0, the first robotics dataset for precision agriculture and forestry addressing the counting and mapping of trees in forestry and orchards. TreeScope provides LiDAR data from agricultural environments collected with robotics platforms, such as UAV and mobile robot platforms carried by vehicles and human operators. In the first release of this dataset, we provide ground-truth data with over 1, 800 manually annotated semantic labels for tree stems and field-measured tree diameters. We share benchmark scripts for these tasks that researchers may use to evaluate the accuracy of their algorithms. Finally, we run our open-source diameter estimation and off-the-shelf semantic segmentation algorithms and share our baseline results. The dataset can be found at https://treescope.org, and the data pre-processing and benchmark code is available at https://github.com/KumarRobotics/treescope.

IROS Conference 2023 Conference Paper

A Data-Driven Approach to Synthesizing Dynamics-Aware Trajectories for Underactuated Robotic Systems

Anusha Srikanthan
Fengjun Yang
Igor Spasojevic
Dinesh Thakur
Vijay Kumar 0001
Nikolai Matni

We consider joint trajectory generation and tracking control for under-actuated robotic systems. A common solution is to use a layered control architecture, where the top layer uses a simplified model of system dynamics for trajectory generation, and the low layer ensures approximate tracking of this trajectory via feedback control. While such layered control architectures are standard and work well in practice, selecting the simplified model used for trajectory generation typically relies on engineering intuition and experience. In this paper, we propose an alternative data-driven approach to dynamicsaware trajectory generation. We show that a suitable augmented Lagrangian reformulation of a global nonlinear optimal control problem results in a layered decomposition of the overall problem into trajectory planning and feedback control layers. Crucially, the resulting trajectory optimization is dynamicsaware, in that, it is modified with a tracking penalty regularizer encoding the dynamic feasibility of the generated trajectory. We show that this tracking penalty regularizer can be learned from system rollouts for independently-designed low layer feedback control policies, and instantiate our framework in the context of a unicycle and a quadrotor control problem in simulation. Further, we show that our approach handles the sim-to-real gap through experiments on the quadrotor hardware platform without any additional training. For both the synthetic unicycle example and the quadrotor system, our framework shows significant improvements in both computation time and dynamic feasibility in simulation and hardware experiments.

ICRA Conference 2023 Conference Paper

Active Metric-Semantic Mapping by Multiple Aerial Robots

Xu Liu 0007
Ankit Prabhu
Fernando Cladera
Ian D. Miller
Lifeng Zhou 0001
Camillo J. Taylor
Vijay Kumar 0001

Traditional approaches for active mapping focus on building geometric maps. For most real-world applications, however, actionable information is related to semantically meaningful objects in the environment. We propose an approach to the active metric-semantic mapping problem that enables multiple heterogeneous robots to collaboratively build a map of the environment. The robots actively explore to minimize the uncertainties in both semantic (object classification) and geometric (object modeling) information. We represent the environment using informative but sparse object models, each consisting of a basic shape and a semantic class label, and characterize uncertainties empirically using a large amount of real-world data. Given a prior map, we use this model to select actions for each robot to minimize uncertainties. The performance of our algorithm is demonstrated through multi-robot experiments in diverse real-world environments. The proposed framework is applicable to a wide range of real-world problems, such as precision agriculture, infrastructure inspection, and asset mapping in factories.

ICRA Conference 2023 Conference Paper

Multi-Robot Coordination and Cooperation with Task Precedence Relationships

Walker Gosrich
Siddharth Mayya
Saaketh Narayan
Matthew Malencia
Saurav Agarwal
Vijay Kumar 0001

We propose a new formulation for the multi-robot task planning and allocation problem that incorporates (a) precedence relationships between tasks; (b) coordination for tasks allowing multiple robots to achieve increased efficiency; and (c) cooperation through the formation of robot coalitions for tasks that cannot be performed by individual robots alone. In our formulation, the tasks and the relationships between the tasks are specified by a task graph. We define a set of reward functions over the task graph's nodes and edges. These functions model the effect of robot coalition size on task performance while incorporating the influence of one task's performance on a dependent task. Solving this problem optimally is NP-hard. However, using the task graph formulation allows us to leverage min-cost network flow approaches to obtain approximate solutions efficiently. Additionally, we explore a mixed integer programming approach, which gives optimal solutions for small instances of the problem but is computationally expensive. We also develop a greedy heuristic algorithm as a baseline. Our modeling and solution approaches result in task plans that leverage task precedence relationships and robot coordination and cooperation to achieve high mission performance, even in large missions with many agents.

IROS Conference 2023 Conference Paper

Robust Localization of Aerial Vehicles via Active Control of Identical Ground Vehicles

Igor Spasojevic
Xu Liu 0007
Ankit Prabhu
Alejandro Ribeiro
George J. Pappas
Vijay Kumar 0001

This paper addresses the problem of active collaborative localization in heterogeneous robot teams with unknown data association. It involves positioning a small number of identical unmanned ground vehicles (UGVs) at desired positions so that an unmanned aerial vehicle (UAV) can, through unlabelled measurements of UGVs, uniquely determine its global pose. We model the problem as a sequential two player game, in which the first player positions the UGVs and the second identifies the two distinct hypothetical poses of the UAV at which the sets of measurements to the UGVs differ by as little as possible. We solve the underlying problem from the vantage point of the first player for a subclass of measurement models using a mixture of local optimization and exhaustive search procedures. Real-world experiments with a team of UAV and UGVs show that our method can achieve centimeter-level global localization accuracy. We also show that our method consistently outperforms random positioning of UGVs by a large margin, with as much as a 90% reduction in position and angular estimation error. Our method can tolerate a significant amount of random as well as non-stochastic measurement noise. This indicates its potential for reliable state estimation on board size, weight, and power (SWaP) constrained UAVs. This work enables robust localization in perceptually-challenged GPS-denied environments, thus paving the road for large-scale multi-robot navigation and mapping.

ICRA Conference 2023 Conference Paper

SEER: Safe Efficient Exploration for Aerial Robots using Learning to Predict Information Gain

Yuezhan Tao
Yuwei Wu 0005
Beiming Li
Fernando Cladera
Alex Zhou
Dinesh Thakur
Vijay Kumar 0001

We address the problem of efficient 3-D exploration in indoor environments for micro aerial vehicles with limited sensing capabilities and payload/power constraints. We develop an indoor exploration framework that uses learning to predict the occupancy of unseen areas, extracts semantic features, samples viewpoints to predict information gains for different exploration goals, and plans informative trajectories to enable safe and smart exploration. Extensive experimentation in simulated and real-world environments shows the proposed approach outperforms the state-of-the-art exploration framework by 24% in terms of the total path length in a structured indoor environment and with a higher success rate during exploration.

ICRA Conference 2023 Conference Paper

Socially Fair Coverage Control

Matthew Malencia
George J. Pappas
Vijay Kumar 0001

We investigate and develop algorithms for social fairness in coverage control problems. Existing coverage control methods are efficient, optimizing the average expected distance from any event to the nearest robot. However, in societal applications like disaster response or transportation, these conventional objectives lead to disparate coverage costs with respect to different groups within a population. We formulate social fairness for coverage control as the minimization of the maximum coverage cost among a set of groups within a population. Our approach uses Voronoi iteration to solve this novel problem by approximating the non-differentiable objective with the log-sum-exp and defining a gradient based controller that prioritizes fairness while also optimizing average performance when disparities between groups are low. We show convergence properties of this proposed control law and demonstrate the approach in simulations of randomly generated population densities as well as environments generated from U. S. census data on population rates and demographics. Our approach provides greater fairness than existing methods while maintaining similar computational time and convergence properties.

ICRA Conference 2023 Conference Paper

The Role of Symmetry in Constructing Geometric Flat Outputs for Free-Flying Robotic Systems

Jake Welde
Matthew D. Kvalheim
Vijay Kumar 0001

Mechanical systems naturally evolve on principal bundles describing their inherent symmetries. The ensuing factorization of the configuration manifold into a symmetry group and an internal shape space has provided deep insights into the locomotion of many robotic and biological systems. On the other hand, the property of differential flatness has enabled efficient, effective planning and control algorithms for various robotic systems. Yet, a practical means of finding a flat output for an arbitrary robotic system remains an open question. In this work, we demonstrate surprising new connections between these two domains, for the first time employing symmetry directly to construct a flat output. We provide sufficient conditions for the existence of a trivialization of the bundle in which the group variables themselves are a flat output. We call this a geometric flat output, since it is equivariant (i. e. it preserves the symmetry) and often global or almost global, properties not typically enjoyed by other flat outputs. In such a trivialization, the motion planning problem is easily solved, since a given trajectory for the group variables will fully determine the trajectory for the shape variables that exactly achieves this motion. We provide a partial catalog of robotic systems with geometric flat outputs and worked examples for the planar rocket, planar aerial manipulator, and quadrotor.

ICRA Conference 2023 Conference Paper

Trajectory Planning for the Bidirectional Quadrotor as a Differentially Flat Hybrid System

Katherine Mao
Jake Welde
M. Ani Hsieh
Vijay Kumar 0001

The use of bidirectional propellers provides quadrotors with greater maneuverability which is advantageous in constrained environments. This paper addresses the development of a trajectory planning algorithm for quadrotors with bidirectional motors. Previous work has shown that the property of differential flatness can be leveraged for efficient trajectory planning. However, planners that leverage flatness for quadrotors fail at points where the acceleration of the center of mass is equal to gravity, i. e. , when the vehicle experiences free fall. The central contribution of this paper is a flatness-based trajectory planning method that allows quadrotors to use bidirectional propellers and pass through the so-called free-fall singularity. We model our system as a differentially flat hybrid system with the aid of coordinate charts derived from the Hopf fibration and develop an algorithm that computes forward and reverse thrusts for each propeller, resulting in smooth trajectories everywhere in SE(3). We demonstrate the planner's versatility by planning knife-edge maneuvers and trajectories passing through the free-fall singularity, while transitioning from forward to reverse thrust.

IROS Conference 2022 Conference Paper

Adaptive Sampling of Latent Phenomena using Heterogeneous Robot Teams (ASLaP-HR)

Matthew Malencia
Sandeep Manjanna
M. Ani Hsieh
George J. Pappas
Vijay Kumar 0001

In this paper, we present an online adaptive planning strategy for a team of robots with heterogeneous sensors to sample from a latent spatial field using a learned model for decision making. Current robotic sampling methods seek to gather information about an observable spatial field. However, many applications, such as environmental monitoring and precision agriculture, involve phenomena that are not directly observable or are costly to measure, called latent phenomena. In our approach, we seek to reason about the latent phenomenon in real-time by effectively sampling the observable spatial fields using a team of robots with heterogeneous sensors, where each robot has a distinct sensor to measure a different observable field. The information gain is estimated using a learned model that maps from the observable spatial fields to the latent phenomenon. This model captures aleatoric uncertainty in the relationship to allow for information theoretic measures. Additionally, we explicitly consider the correlations among the observable spatial fields, capturing the relationship between sensor types whose observations are not independent. We show it is possible to learn these correlations, and investigate the impact of the learned correlation models on the performance of our sampling approach. Through our qualitative and quantitative results, we illustrate that empirically learned correlations improve the overall sampling efficiency of the team. We simulate our approach using a data set of sensor measurements collected on Lac Hertel, in Quebec, which we make publicly available.

ICRA Conference 2022 Conference Paper

Coverage Control in Multi-Robot Systems via Graph Neural Networks

Walker Gosrich
Siddharth Mayya
Rebecca Li
James Paulos
Mark Yim
Alejandro Ribeiro
Vijay Kumar 0001

This paper develops a decentralized approach to mobile sensor coverage by a multi-robot system. We consider a scenario where a team of robots with limited sensing range must position itself to effectively detect events of interest in a region characterized by areas of varying importance. Towards this end, we develop a decentralized control policy for the robots-realized via a Graph Neural Network-which uses inter-robot communication to leverage non-local information for control decisions. By explicitly sharing information between multi-hop neighbors, the decentralized controller achieves a higher quality of coverage when compared to classical approaches that do not communicate and leverage only local information available to each robot. Simulated experiments demonstrate the efficacy of multi-hop communication for multi-robot coverage and evaluate the scalability and transferability of the learning-based controllers.

ICRA Conference 2022 Conference Paper

Experiments in Adaptive Replanning for Fast Autonomous Flight in Forests

Laura Jarin-Lipschitz
Xu Liu 0007
Yuezhan Tao
Vijay Kumar 0001

Fast, autonomous flight in unstructured, cluttered environments such as forests is challenging because it requires the robot to compute new plans in realtime on a computationally-constrained platform. In this paper, we enable this capability with a search-based planning framework that adapts sampling density in realtime to find dynamically-feasible plans while remaining computationally tractable. A paramount challenge in search-based planning is that dense obstacles both necessitate large graphs (to guarantee completeness) and reduce the efficiency of graph search (as heuristics become less accurate). To address this, we develop a planning framework with two parts: one that maximizes planner completeness for a given graph size, and a second that dynamically maximizes graph size subject to computational constraints. This framework is enabled by motion planning graphs that are defined by a single parameter-dispersion-which quantifies the maximum trajectory cost to reach an arbitrary state from the graph. We show through real and simulated experiments how the dispersion can be adapted to different environments in realtime, allowing operation in environments with varying density. The simulated experiment demonstrates improved performance over a baseline search-based planning algorithm. We also demonstrate flight speeds of up to 2. 5m/s in real-world cluttered pine forests.

ICRA Conference 2022 Conference Paper

Robust Semantic Mapping and Localization on a Free-Flying Robot in Microgravity

Ian D. Miller
Ryan Soussan
Brian Coltin
Trey Smith
Vijay Kumar 0001

We propose a system that uses semantic object detections to localize a microgravity free-flyer. Many applications require absolute localization in a known reference frame, such as the execution of waypoint trajectories defined by human operators. Classical geometric methods build a map of point features, which may not be able to be associated after lighting or environmental changes. By contrast, semantics remain invariant to changes up to the robustness of the detection algorithm and motion of the semantic objects. In this work, we describe our approaches for both offline semantic map generation as well as online localization against a semantic map, intended to run in real-time on the robot. We additionally demonstrate how our semantic localizer outperforms image-feature matching in some cases, and show the robustness of the algorithm to environmental changes. Crucially, we show in our experiments that when semantics are used to supplement point features, localization is always improved. To our knowledge, these experiments demonstrate the first use of learned semantics for localization on a free-flying robot in microgravity.

ICRA Conference 2022 Conference Paper

RTGNN: A Novel Approach to Model Stochastic Traffic Dynamics

Ke Sun 0008
Stephen Chaves
Paul Martin 0012
Vijay Kumar 0001

Modeling stochastic traffic dynamics is critical to developing self-driving cars. Because it is difficult to develop first principle models of cars driven by humans, there is great potential for using data driven approaches in developing traffic dynamical models. While there is extensive literature on this subject, previous works mainly address the prediction accuracy of data-driven models. Moreover, it is often difficult to apply these models to common planning frameworks since they fail to meet the assumptions therein. In this work, we propose a new stochastic traffic model, Recurrent Traffic Graph Neural Network (RTGNN), by enforcing additional structures on the model so that the proposed model can be seamlessly integrated with existing motion planning algorithms. RTGNN is a Markovian model and is able to infer future traffic states conditioned on the motion of the ego vehicle. Specifically, RTGNN uses a definition of the traffic state that includes the state of all players in a local region and is therefore able to make joint predictions for all agents of interest. Meanwhile, we explicitly model the hidden states of agents, “intentions, ” as part of the traffic state to reflect the inherent partial observability of traffic dynamics. The above mentioned properties are critical for integrating RTGNN with motion planning algorithms coupling prediction and decision making. Despite the additional structures, we show that RTGNN is able to achieve state-of-the-art accuracy through comparisons with other similar works.

IROS Conference 2021 Conference Paper

Combined Routing and Scheduling of Heterogeneous Transport and Service Agents

Saaketh Narayan
James Paulos
Steven W. Chen
Sandeep Manjanna
Vijay Kumar 0001

This paper investigates servicing waypoints in a wide area using collaborative deployments of vehicles with heterogeneous range and mobility constraints. We formulate a joint planning problem for a single transport truck and multiple service drones in which the truck is constrained to a road and must deploy a team of range-constrained drones to visit waypoints. The need to deploy, collect, and redeploy drones over multiple flights introduces both route finding and scheduling aspects to this problem. We solve large problem instances by decoupling our approach into a service drone route finding phase and a transport truck scheduling phase. Numerical simulations explore the qualitative character of the driving schedule and the quantitative marginal value of adding additional drones to the team as a function of agent number and relative speed. The combination of road network constraints and range constraints make this problem especially relevant to wide area forestry, last-mile delivery, and ecological monitoring applications.

ICRA Conference 2021 Conference Paper

Dispersion-Minimizing Motion Primitives for Search-Based Motion Planning

Laura Jarin-Lipschitz
James Paulos
Raymond Bjorkman
Vijay Kumar 0001

Search-based planning with motion primitives is a powerful motion planning technique that can provide dynamic feasibility, optimality, and real-time computation times on size, weight, and power-constrained platforms in unstructured environments. However, optimal design of the motion planning graph, while crucial to the performance of the planner, has not been a main focus of prior work. This paper proposes to address this by introducing a method of choosing vertices and edges in a motion primitive graph that is grounded in sampling theory and leads to theoretical guarantees on planner completeness. By minimizing dispersion of the graph vertices in the metric space induced by trajectory cost, we optimally cover the space of feasible trajectories with our motion primitive graph. In comparison with baseline motion primitives defined by uniform input space sampling, our motion primitive graphs have lower dispersion, find a plan with fewer iterations of the graph search, and have only one parameter to tune.

IROS Conference 2021 Conference Paper

Learning Connectivity for Data Distribution in Robot Teams

Ekaterina I. Tolstaya
Landon Butler
Daniel Mox
James Paulos
Vijay Kumar 0001
Alejandro Ribeiro

Many algorithms for control of multi-robot teams operate under the assumption that low-latency, global state information necessary to coordinate agent actions can readily be disseminated among the team. However, in harsh environments with no existing communication infrastructure, robots must form ad-hoc networks, forcing the team to operate in a distributed fashion. To overcome this challenge, we propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN). Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. To do this, agents glean information about the topology of the network from packet transmissions and feed it to a GNN running locally which instructs the agent when and where to transmit the latest state information. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions. Our approach performs favorably compared to industry-standard methods for data distribution such as random flooding and round robin. We also show that the trained policies generalize to larger teams of both static and mobile agents.

IROS Conference 2021 Conference Paper

Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks

Ekaterina I. Tolstaya
James Paulos
Vijay Kumar 0001
Alejandro Ribeiro

The multi-robot coverage problem is an essential building block for systems that perform tasks like inspection, exploration, or search and rescue. We discretize the coverage problem to induce a spatial graph of locations and represent robots as nodes in the graph. Then, we train a Graph Neural Network controller that leverages the spatial equivariance of the task to imitate an expert open-loop routing solution. This approach generalizes well to much larger maps and larger teams that are intractable for the expert. In particular, the model generalizes effectively to a simulation of ten quadrotors and dozens of buildings in an urban setting. We also demonstrate the GNN controller can surpass planning-based approaches in an exploration task.

IROS Conference 2020 Conference Paper

Adaptive Partitioning for Coordinated Multi-agent Perimeter Defense

Douglas G. Macharet
Austin K. Chen
Daigo Shishika
George J. Pappas
Vijay Kumar 0001

Multi-Robot Systems have been recently employed in different applications and have advantages over single-robot systems, such as increased robustness and task performance efficiency. We consider such assemblies specifically in the scenario of perimeter defense, where the task is to defend a circular perimeter by intercepting radially approaching targets. Possible intruders appear randomly at a fixed distance from the perimeter and with azimuthal location determined by some unknown probability density. Coordination among multiple defenders is a complex combinatorial optimization problem. In this work, we focus on the following two aspects: (i) estimating the probability density that describes the direction from which the next intruders are going to arrive, and (ii) partitioning of the space so that the defenders focus on capturing a disjoint subset of intruders. Results show that the proposed strategy increases the number of captures over a naive baseline strategy, especially in scenarios with non-uniform spatial distributions of intruder arrival. The proposed approach is also efficient and able to quickly adapt to time-varying intruder distributions.

ICRA Conference 2020 Conference Paper

Coordinate-Free Dynamics and Differential Flatness of a Class of 6DOF Aerial Manipulators

Jake Welde
Vijay Kumar 0001

In this work, we derive a coordinate-free formulation of the coupled dynamics of a class of 6DOF aerial manipulators consisting of an underactuated quadrotor equipped with a 2DOF articulated manipulator, and demonstrate that the system is differentially flat with respect to the end effector pose. In particular, we require the center of mass of the entire system to be fixed in the end effector frame, suggesting a reasonable mechanical design criterion. We make use of an inertial decoupling transformation to demonstrate differential flatness, allowing us to plan dynamically feasible trajectories for the system in the space of the 6DOF pose of the end effector, which is ideal for achieving precise manipulator tasks. Simulation results validate the flatness-based planning methodology for our dynamic model, and its usefulness is demonstrated in a simulated aerial videography task.

ICRA Conference 2020 Conference Paper

DC-CAPT: Concurrent Assignment and Planning of Trajectories for Dubins Cars

Michael Whitzer
Daigo Shishika
Dinesh Thakur
Vijay Kumar 0001
Amanda Prorok

We present an algorithm for the concurrent assignment and planning of collision-free trajectories (DC-CAPT) for robots whose kinematics can be modeled as Dubins cars, i. e. , robots constrained in terms of their initial orientation and their minimum turning radius. Coupling the assignment and trajectory planning subproblems allows for a computationally tractable solution. This solution is guaranteed to be collision- free through the use of a single constraint: the start and goal locations have separation distance greater than some threshold. We derive this separation distance by extending a prior work that assumed holonomic robots. We demonstrate the validity of our approach, and show its efficacy through simulations and experiments where groups of robots executing Dubins curves travel to their assigned goal locations without collisions.

ICRA Conference 2020 Conference Paper

Dense r-robust formations on lattices

Luis Guerrero-Bonilla
David Saldaña
Vijay Kumar 0001

Robot networks are susceptible to fail under the presence of malicious or defective robots. Resilient networks in the literature require high connectivity and large communication ranges, leading to high energy consumption in the communication network. This paper presents robot formations with guaranteed resiliency that use smaller communication ranges than previous results in the literature. The formations can be built on triangular and square lattices in the plane, and cubic lattices in the three-dimensional space. We support our theoretical framework with simulations.

IROS Conference 2020 Conference Paper

Feedback Enhanced Motion Planning for Autonomous Vehicles

Ke Sun 0008
Brent Schlotfeldt
Stephen Chaves
Paul Martin 0012
Gulshan Mandhyan
Vijay Kumar 0001

In this work, we address the motion planning problem for autonomous vehicles through a new lattice planning approach, called Feedback Enhanced Lattice Planner (FELP). Existing lattice planners have two major limitations, namely the high dimensionality of the lattice and the lack of modeling of agent vehicle behaviors. We propose to apply the Intelligent Driver Model (IDM) [1] as a speed feedback policy to address both of these limitations. IDM both enables the responsive behavior of the agents, and uniquely determines the acceleration and speed profile of the ego vehicle on a given path. Therefore, only a spatial lattice is needed, while discretization of higher order dimensions is no longer required. Additionally, we propose a directed-graph map representation to support the implementation and execution of lattice planners. The map can reflect local geometric structure, embed the traffic rules adhering to the road, and is efficient to construct and update. We show that FELP is more efficient compared to other existing lattice planners through runtime complexity analysis, and we propose two variants of FELP to further reduce the complexity to polynomial time. We demonstrate the improvement by comparing FELP with an existing spatiotemporal lattice planner using simulations of a merging scenario and continuous highway traffic. We also study the performance of FELP under different traffic densities.

IROS Conference 2020 Conference Paper

Game Theoretic Formation Design for Probabilistic Barrier Coverage

Daigo Shishika
Douglas G. Macharet
Brian M. Sadler
Vijay Kumar 0001

We study strategies to deploy defenders/sensors to detect intruders that approach a targeted region. This scenario is formulated as a barrier coverage, which aims to minimize the number of unseen paths. The problem becomes challenging when the number of defenders is insufficient for a full coverage, requiring us to find the most effective location to deploy them. To this end, we use ideas from game theory to account for various paths that the intruders may take. Specifically, we propose an iterative algorithm to refine the set of candidate defender formations, which uses the payoff matrix to directly evaluate the utility of different formations. Given the set of candidate formations, a mixed Nash equilibrium gives a stochastic policy to deploy the defenders. The efficacy of the proposed strategy is demonstrated by a numerical analysis that compares our method with an existing graph-theoretic method.

ICRA Conference 2020 Conference Paper

Information Theoretic Active Exploration in Signed Distance Fields

Kelsey Saulnier
Nikolay Atanasov 0001
George J. Pappas
Vijay Kumar 0001

This paper focuses on exploration and occupancy mapping of unknown environments using a mobile robot. While a truncated signed distance field (TSDF) is a popular, efficient, and highly accurate representation of occupancy, few works have considered optimizing robot sensing trajectories for autonomous TSDF mapping. We propose an efficient approach for maintaining TSDF uncertainty and predicting its evolution from potential future sensor measurements without actually receiving them. Efficient uncertainty prediction is critical for long-horizon optimization of potential sensing trajectories. We develop a deterministic tree-search algorithm that evaluates the information gain between the TSDF distribution and potential observations along sequences of robot motion primitives. Efficient planning is achieved by branch-and-bound pruning of uninformative sensing trajectories. The effectiveness of our active TSDF mapping approach is evaluated in several simulated environments with complex visibility constraints.

ICRA Conference 2020 Conference Paper

Mobile Wireless Network Infrastructure on Demand

Daniel Mox
Miguel Calvo-Fullana
Mikhail Gerasimenko
Jonathan Fink
Vijay Kumar 0001
Alejandro Ribeiro

In this work, we introduce Mobile Wireless Infrastructure on Demand: a framework for providing wireless connectivity to multi-robot teams via autonomously reconfiguring ad-hoc networks. In many cases, previous multi-agent systems either assumed the availability of existing communication infrastructure or were required to create a network in addition to completing their objective. Instead our system explicitly assumes the responsibility of creating and sustaining a wireless network capable of satisfying end-to-end communication requirements of a team of agents, called the task team, performing an arbitrary objective. To accomplish this goal, we propose a joint optimization framework that alternates between finding optimal network routes to support data flows between the task agents and improving the performance of the network by repositioning a collection of mobile relay nodes referred to as the network team. We demonstrate our approach with simulations and experiments wherein wireless connectivity is provided to patrolling task agents.

ICRA Conference 2020 Conference Paper

PST900: RGB-Thermal Calibration, Dataset and Segmentation Network

Shreyas S. Shivakumar
Neil Rodrigues
Alex Zhou
Ian D. Miller
Vijay Kumar 0001
Camillo J. Taylor

In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean Challenge. Lastly, we propose a CNN architecture for fast semantic segmentation that combines both RGB and Thermal imagery in a way that leverages RGB imagery independently. We compare our method against the state-of-the-art and show that our method outperforms them in our dataset.

ICRA Conference 2020 Conference Paper

Reactive Temporal Logic Planning for Multiple Robots in Unknown Environments

Yiannis Kantaros
Matthew Malencia
Vijay Kumar 0001
George J. Pappas

This paper proposes a new reactive mission planning algorithm for multiple robots that operate in unknown environments. The robots are equipped with individual sensors that allow them to collectively learn and continuously update a map of the unknown environment. The goal of the robots is to accomplish complex tasks, captured by global co-safe Linear Temporal Logic (LTL) formulas. The majority of existing temporal logic planning approaches rely on discrete abstractions of the robot dynamics operating in known environments and, as a result, they cannot be applied to the more realistic scenarios where the environment is initially unknown. In this paper, we address this novel challenge by proposing the first reactive, and abstraction-free LTL planning algorithm that can be applied for complex mission planning of multiple robots operating in unknown environments. Our algorithm is reactive in the sense that temporal logic planning is adapting to the updated map of the environment and abstraction-free as it does not rely on designing abstractions of robot dynamics. Our proposed algorithm is complete under mild assumptions on the structure of the environment and the sensor models. Our paper provides extensive numerical simulations and hardware experiments that illustrate the theoretical analysis and show that the proposed algorithm can address complex planning tasks in unknown environments.

IROS Conference 2020 Conference Paper

Robust, Perception Based Control with Quadrotors

Laura Jarin-Lipschitz
Rebecca Li
Ty Nguyen
Vijay Kumar 0001
Nikolai Matni

Traditionally, controllers and state estimators in robotic systems are designed independently. Controllers are often designed assuming perfect state estimation. However, state estimation methods such as Visual Inertial Odometry (VIO) drift over time and can cause the system to misbehave. While state estimation error can be corrected with the aid of GPS or motion capture, these complementary sensors are not always available or reliable. Recent work has shown that this issue can be dealt with by synthesizing robust controllers using a data-driven characterization of the perception error, and can bound the system's response to state estimation error using a robustness constraint. We investigate the application of this robust perception-based approach to a quadrotor model using VIO for state estimation and demonstrate the benefits and drawbacks of using this technique in simulation and hardware. Additionally, to make tuning easier, we introduce a new cost function to use in the control synthesis which allows one to take an existing controller and "robustify" it. To the best of our knowledge, this is the first robust perception-based controller implemented in real hardware, as well as one utilizing a data-driven perception model. We believe this as an important step towards safe, robust robots that explicitly account for the inherent dependence between perception and control.

ICRA Conference 2020 Conference Paper

The Tiercel: A novel autonomous micro aerial vehicle that can map the environment by flying into obstacles

Yash Mulgaonkar
Wenxin Liu 0002
Dinesh Thakur
Kostas Daniilidis
Camillo J. Taylor
Vijay Kumar 0001

Autonomous flight through unknown environments in the presence of obstacles is a challenging problem for micro aerial vehicles (MAVs). A majority of the current state-of-art research assumes obstacles as opaque objects that can be easily sensed by optical sensors such as cameras or LiDARs. However in indoor environments with glass walls and windows, or scenarios with smoke and dust, robots (even birds) have a difficult time navigating through the unknown space. In this paper, we present the design of a new class of micro aerial vehicles that achieves autonomous navigation and are robust to collisions. In particular, we present the Tiercel MAV: a small, agile, light weight and collision-resilient robot powered by a cellphone grade CPU. Our design exploits contact to infer the presence of transparent or reflective obstacles like glass walls, integrating touch with visual perception for SLAM. The Tiercel is able to localize using visual-inertial odometry (VIO) running on board the robot with a single downward facing fisheye camera and an IMU. We show how our collision detector design and experimental set up enable us to characterize the impact of collisions on VIO. We further develop a planning strategy to enable the Tiercel to fly autonomously in an unknown space, sustaining collisions and creating a 2D map of the environment. Finally we demonstrate a swarm of three autonomous Tiercel robots safely navigating and colliding through an obstacle field to reach their objectives.

ICRA Conference 2020 Conference Paper

Vision-based Multi-MAV Localization with Anonymous Relative Measurements Using Coupled Probabilistic Data Association Filter

Ty Nguyen
Kartik Mohta
Camillo J. Taylor
Vijay Kumar 0001

We address the localization of robots in a multi-MAV system where external infrastructure like GPS or motion capture systems may not be available. Our approach lends itself to implementation on platforms with several constraints on size, weight, and power (SWaP). Particularly, our framework fuses the onboard VIO with the anonymous, visual-based robot-to-robot detection to estimate all robot poses in one common frame, addressing three main challenges: 1) the initial configuration of the robot team is unknown, 2) the data association between each vision-based detection and robot targets is unknown, and 3) the vision-based detection yields false negatives, false positives, inaccurate, and provides noisy bearing, distance measurements of other robots. Our approach extends the Coupled Probabilistic Data Association Filter [1] to cope with nonlinear measurements. We demonstrate the superior performance of our approach over a simple VIO-based method in a simulation with the measurement models statistically modeled using the real experimental data. We also show how onboard sensing, estimation, and control can be used for formation flight.

ICRA Conference 2019 Conference Paper

Decentralization of Multiagent Policies by Learning What to Communicate

James Paulos
Steven W. Chen
Daigo Shishika
Vijay Kumar 0001

Effective communication is required for teams of robots to solve sophisticated collaborative tasks. In practice it is typical for both the encoding and semantics of communication to be manually defined by an expert; this is true regardless of whether the behaviors themselves are bespoke, optimization based, or learned. We present an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy. A perimeter defense game illustrates the system's ability to handle dynamically changing numbers of agents and its graceful degradation in performance as communication constraints are tightened or the expert's observability assumptions are broken.

IROS Conference 2019 Conference Paper

Inverse Optimal Planning for Air Traffic Control

Ekaterina I. Tolstaya
Alejandro Ribeiro
Vijay Kumar 0001
Ashish Kapoor

We envision a system that concisely describes the rules of air traffic control, assists human operators and supports dense autonomous air traffic around commercial airports. We develop a method to learn the rules of air traffic control from real data as a cost function via maximum entropy inverse reinforcement learning. This cost function is used as a penalty for a search-based motion planning method that discretizes both the control and the state space. We illustrate the methodology by showing that our approach can learn to imitate the airport arrival routes and separation rules of dense commercial air traffic. The resulting trajectories are shown to be safe, feasible, and efficient.

IROS Conference 2019 Conference Paper

Learning Safe Unlabeled Multi-Robot Planning with Motion Constraints

Arbaaz Khan
Chi Zhang
Shuo Li
Jiayue Wu
Brent Schlotfeldt
Sarah Y. Tang
Alejandro Ribeiro
Osbert Bastani

In this paper, we present a learning approach to goal assignment and trajectory planning for unlabeled robots operating in 2D, obstacle-filled workspaces. More specifically, we tackle the unlabeled multi-robot motion planning problem with motion constraints as a multi-agent reinforcement learning problem with some sparse global reward. In contrast with previous works, which formulate an entirely new hand-crafted optimization cost or trajectory generation algorithm for a different robot dynamic model, our framework is a general approach that is applicable to arbitrary robot models. Further, by using the velocity obstacle, we devise a smooth projection that guarantees collision free trajectories for all robots with respect to their neighbors and obstacles. The efficacy of our algorithm is demonstrated through varied simulations. A video describing our method and results can be found here.

ICRA Conference 2019 Conference Paper

ModQuad-Vi: A Vision-Based Self-Assembling Modular Quadrotor

Guanrui Li
Bruno Gabrich
David Saldaña
Jnaneshwar Das
Vijay Kumar 0001
Mark Yim

Flying modular robots have the potential to rapidly form temporary structures. In the literature, docking actions rely on external systems and indoor infrastructures for relative pose estimation. In contrast to related work, we provide local estimation during the self-assembly process to avoid dependency on external systems. In this paper, we introduce ModQuad-Vi, a flying modular robot that is aimed to operate in outdoor environments. We propose a new robot design and vision-based docking method. Our design is based on a quadrotor platform with onboard computation and visual perception. Our control method is able to accurately align modules for docking actions. Additionally, we present the dynamics and a geometric controller for the aerial modular system. Experiments validate the vision-based docking method with successful results.

ICRA Conference 2019 Conference Paper

Online Estimation of Geometric and Inertia Parameters for Multirotor Aerial Vehicles

Valentin Wüest
Vijay Kumar 0001
Giuseppe Loianno

Accurate knowledge of geometric and inertia parameters are a necessity for precise and robust control of aerial vehicles. We propose a novel filter that is able to fuse motor speed, inertia, and pose measurements to estimate the vehicle's key dynamic properties online. The presented framework is able to estimate the multirotor's moment of inertia, mass, center of mass and each sensor module's relative position. Obtaining these estimates in-flight allow the multirotor to be precisely controlled even during tasks such as load transportation or after configuration changes on scene. We provide a nonlinear observability analysis, proving that the presented model is locally weakly observable. Experimental results validate the proposed approach, showing the ability to estimate the dynamic properties accurately and demonstrate its capability to do so even while additional loads are added. The framework is flexible and can easily be adapted to a wide range of applications, including self-calibration, object grasping, and single robot or multi-robot payload transportation.

ICRA Conference 2019 Conference Paper

Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements

Shreyas S. Shivakumar
Kartik Mohta
Bernd Pfrommer
Vijay Kumar 0001
Camillo J. Taylor

We present an approach to depth estimation that fuses information from a stereo pair with sparse range measurements derived from a LIDAR sensor or a range camera. The goal of this work is to exploit the complementary strengths of the two sensor modalities, the accurate but sparse range measurements and the ambiguous but dense stereo information. These two sources are effectively and efficiently fused by combining ideas from anisotropic diffusion and semi-global matching. We evaluate our approach on the KITTI 2015 and Middlebury 2014 datasets, using randomly sampled ground truth range measurements as our sparse depth input. We achieve significant performance improvements with a small fraction of range measurements on both datasets. We also provide qualitative results from our platform using the PMDTec Monstar sensor. Our entire pipeline runs on an NVIDIA TX-2 platform at 5Hz on 1280×1024 stereo images with 128 disparity levels.

ICRA Conference 2019 Conference Paper

The Open Vision Computer: An Integrated Sensing and Compute System for Mobile Robots

Morgan Quigley
Kartik Mohta
Shreyas S. Shivakumar
Michael Watterson
Yash Mulgaonkar
Mikael Arguedas
Ke Sun 0008
Sikang Liu 0002

In this paper we describe the Open Vision Computer (OVC) which was designed to support high speed, vision guided autonomous drone flight. In particular our aim was to develop a system that would be suitable for relatively small-scale flying platforms where size, weight, power consumption and computational performance were all important considerations. This manuscript describes the primary features of our OVC system and explains how they are used to support fully autonomous indoor and outdoor exploration and navigation operations on our Falcon 250 quadrotor platform.

ICRA Conference 2018 Conference Paper

A Flying Gripper Based on Cuboid Modular Robots

Bruno Gabrich
David Saldaña
Vijay Kumar 0001
Mark Yim

We present a novel flying modular platform capable of grasping and transporting objects. It is composed of four cooperative identical modules where each is based on a quadrotor within a cuboid frame with a docking mechanism. Pairs of modules are able to fly independently and physically connect by matching their vertical edges forming a hinge. Four one degree of freedom (DOF) connections results in a one DOF four-bar linkage that can be used to grasp external objects. In this paper, we propose a decentralized method that allows the Flying Gripper to control its position, attitude and aperture angle. In our experiments, we tested the hovering performance for different aperture angles and with a grasped object. The performance for a closing and opening motion was also verified.

ICRA Conference 2018 Conference Paper

Experiments in Fast, Autonomous, GPS-Denied Quadrotor Flight

Kartik Mohta
Ke Sun 0008
Sikang Liu 0002
Michael Watterson
Bernd Pfrommer
James Svacha
Yash Mulgaonkar
Camillo J. Taylor

High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to fly at speeds of more than 18 m/s. We provide an overview of our system and details about the specific component technologies that enable the high speed navigation capability of our platform. We demonstrate the robustness of our system through high speed autonomous flights and navigation through a variety of obstacle rich environments.

ICRA Conference 2018 Conference Paper

Human Motion Capture Using a Drone

Xiaowei Zhou 0001
Sikang Liu 0002
Georgios Pavlakos
Vijay Kumar 0001
Kostas Daniilidis

Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficial for motion reconstruction. We evaluate the accuracy of the proposed system using our new DroCap dataset and also demonstrate its applicability for MoCap in the wild using a consumer drone.

IROS Conference 2018 Conference Paper

Inertial Velocity and Attitude Estimation for Quadrotors

James Svacha
Kartik Mohta
Michael Watterson
Giuseppe Loianno
Vijay Kumar 0001

This work addresses the design and implementation of a filter that estimates the orientation of the body-fixed z axis and the velocity of a quadrotor UAV from the inertial measurement unit (IMU) given a known yaw. The velocity and attitude estimation is possible since the filter employs a linear drag model measuring the drag forces on the quadrotor through the IMU. These forces are functions of the robot's velocity and attitude. In addition, the filter estimates the linear drag parameters and thrust coefficient for the propellers. These parameters may be fed back into a controller to improve tracking performance. Experimental results are used to validate the proposed approach.

IROS Conference 2018 Conference Paper

Learning Sample-Efficient Target Reaching for Mobile Robots

Arbaaz Khan
Vijay Kumar 0001
Alejandro Ribeiro

In this paper, we propose a novel architecture and a self-supervised policy gradient algorithm, which employs unsupervised auxiliary tasks to enable a mobile robot to learn how to navigate to a given goal. The dependency on the global information is eliminated by providing only sparse range-finder measurements to the robot. The partially observable planning problem is addressed by splitting it into a hierarchical process. We use convolutional networks to plan locally, and a differentiable memory to provide information about past time steps in the trajectory. These modules, combined in our network architecture, produce globally consistent plans. The sparse reward problem is mitigated by our modified policy gradient algorithm. We model the robots uncertainty with unsupervised tasks to force exploration. The novel architecture we propose with the modified version of the policy gradient algorithm allows our robot to reach the goal in a sample efficient manner, which is orders of magnitude faster than the current state of the art policy gradient algorithm. Simulation and experimental results are provided to validate the proposed approach.

IROS Conference 2018 Conference Paper

Model Predictive Trajectory Tracking and Collision Avoidance for Reliable Outdoor Deployment of Unmanned Aerial Vehicles

Tomás Báca
Daniel Hert
Giuseppe Loianno
Martin Saska
Vijay Kumar 0001

We propose a novel approach for optimal trajectory tracking for unmanned aerial vehicles (UAV), using a linear model predictive controller (MPC) in combination with non-linear state feedback. The solution relies on fast onboard simulation of the translational dynamics of the UAV, which is guided by a linear MPC. By sampling the states of the virtual UAV, we create a control command for fast non-linear feedback, which is capable of performing agile maneuvers with high precision. In addition, the proposed pipeline provides an interface for a decentralized collision avoidance system for multi-UAY scenarios. Our solution makes use of the long prediction horizon of the linear MPC and allows safe outdoors execution of multi-UAV experiments without the need for in-advance collision-free planning. The practicality of the tracking mechanism is shown in combination with priority-based collision resolution strategy, which performs sufficiently in experiments with up to 5 UAVs. We present a statistical and experimental evaluation of the platform in both simulation and real-world examples, demonstrating the usability of the approach.

ICRA Conference 2018 Conference Paper

ModQuad: The Flying Modular Structure that Self-Assembles in Midair

David Saldaña
Bruno Gabrich
Guanrui Li
Mark Yim
Vijay Kumar 0001

We introduce ModQuad, a novel flying modular robotic structure that is able to self-assemble in midair and cooperatively fly. The structure is composed by agile flying modules that can easily move in a three dimensional environment. The module is based on a quadrotor platform within a cuboid frame which allows it to attach to other modules by matching vertical faces. Using this mechanism, a ModQuad swarm is able to rapidly assemble flying structures in midair using the robot bodies as building units. In this paper, we focus on two important tasks for modular flying structures. First, we propose a decentralized modular attitude controller to allow a team of physically connected modules to fly cooperatively. Second, we develop a docking method that drives pairs of structures to be attached in midair. Our method precisely aligns, and corrects motion errors during the docking process. In our experiments, we tested and analyzed the performance of the cooperative flying method for multiple configurations. We also tested the docking method with successful results.

IROS Conference 2018 Conference Paper

Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion

Xu Liu 0007
Steven W. Chen
Shreyas Aditya
Nivedha Sivakumar
Sandeep Dcunha
Chao Qu
Camillo J. Taylor
Jnaneshwar Das

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.

ICRA Conference 2018 Conference Paper

Semi-Dense Visual-Inertial Odometry and Mapping for Quadrotors with SWAP Constraints

Wenxin Liu 0002
Giuseppe Loianno
Kartik Mohta
Kostas Daniilidis
Vijay Kumar 0001

Micro Aerial Vehicles have the potential to assist humans in real life tasks involving applications such as smart homes, search and rescue, and architecture construction. To enhance autonomous navigation capabilities these vehicles need to be able to create dense 3D maps of the environment, while concurrently estimating their own motion. In this paper, we are particularly interested in small vehicles that can navigate cluttered indoor environments. We address the problem of visual inertial state estimation, control and 3D mapping on platforms with Size, Weight, And Power (SWAP) constraints. The proposed approach is validated through experimental results on a 250 g, 22 cm diameter quadrotor equipped only with a stereo camera and an IMU with a computationally-limited CPU showing the ability to autonomously navigate, while concurrently creating a 3D map of the environment.

IROS Conference 2017 Conference Paper

A decentralized algorithm for assembling structures with modular robots

David Saldaña
Bruno Gabrich
Michael Whitzer
Amanda Prorok
Mario F. M. Campos
Mark Yim
Vijay Kumar 0001

Recent work in the field of bio-inspired robotic systems has introduced designs for modular robots that are able to assemble into structures (e. g. , bridges, landing platforms, fences) using their bodies as the building components. Yet, it remains an open question as to how to program large swarms of robotic modules so that the assembly task is performed as efficiently as possible. Moreover, the problem of designing assembly algorithms is compounded by the scale of these systems, and by the lack of centralized guidance in unstructured environments. The main contribution of this work is a decentralized algorithm to assemble structures with modular robots. Importantly, we coordinate the robots so that docking actions can be parallelized. We show the correctness of our algorithm, and we demonstrate its scalability and generality through multiple scenarios in simulation. Experiments on physical robots demonstrate the validity of our approach in real-world settings.

IROS Conference 2017 Conference Paper

Intrusion detection for stochastic task allocation in robot swarms

Florian Maushart
Amanda Prorok
M. Ani Hsieh
Vijay Kumar 0001

We present a novel framework for integrity analysis of swarm robotic systems using the symmetric Kullback-Leibler Divergence. The objective is to understand a robot swarm's vulnerability to malicious intrusion and to develop the necessary computational tools that would detect the presence of malicious agents within the swarm. Using ensemble approaches for modeling and analyzing stochastic task allocation, we analyze the performance of the proposed strategy subject to different system parameters, and show how different design choices can facilitate early intrusion detection. We further evaluate the performance of our method in realistic scenarios through stochastic simulations for different team sizes. The main contribution is an analysis framework whose output can be used to avoid system-inherent design flaws and to decrease the damage that can be inflicted by an undetected attacker.

IROS Conference 2017 Conference Paper

Precise dispensing of liquids using visual feedback

Monroe Kennedy III
Kendall Queen
Dinesh Thakur
Kostas Daniilidis
Vijay Kumar 0001

Robotic pouring is an important step in improving the safety, productivity and repeatability in the biotechnology industry and generally increasing the effectiveness of robotics in human based environments. In this work we present a method to autonomously dispense a precise amount of fluid using only visual feedback without using precision pouring instruments such as pipettes, syringes or pourers. We model circular and rectangular pouring container geometries. We prove that for square containers we can control the flow by only observing the fluid height in the receiving beaker. We show a systematic approach using a hybrid control scheme that is robust to the initial amount of fluid in the pouring container and inconsistent flow. Specifically we present (a) a model for pouring (b) a model based algorithm to drive a robot arm (c) visual feedback for regulating the pouring rate. We demonstrate this using the Rethink Robotics Sawyer manipulator and mvBluefox MLC202bc camera.

IROS Conference 2017 Conference Paper

Privacy-preserving vehicle assignment for mobility-on-demand systems

Amanda Prorok
Vijay Kumar 0001

Urban transportation is being transformed by mobility-on-demand (MoD) systems. One of the goals of MoD systems is to provide personalized transportation services to passengers. This process is facilitated by a centralized operator that coordinates the assignment of vehicles to individual passengers, based on location data. However, current approaches assume that accurate positioning information for passengers and vehicles is readily available. This assumption raises privacy concerns. In this work, we address this issue by proposing a method that protects passengers' drop-off locations (i. e. , their travel destinations). Formally, we solve a batch assignment problem that routes vehicles at obfuscated origin locations to passenger locations (since origin locations correspond to previous drop-off locations), such that the mean waiting time is minimized. Our main contributions are two-fold. First, we formalize the notion of privacy for continuous vehicle-to-passenger assignment in MoD systems, and integrate a privacy mechanism that provides formal guarantees. Second, we present a polynomial-time iterative version of the Hungarian algorithm to allocate a redundant number of vehicles to a single passenger. This algorithm builds on the insight that even during peak rush hour there are unoccupied (redundant) traveling vehicles. This strategy allows us to reduce the performance deterioration induced by the privacy mechanism. In particular, it enables the exploration of the trade-off between privacy levels, waiting time, and deployed fleet size. We evaluate our methods on a real, large-scale data set consisting of over 11 million taxi rides (specifying vehicle availability and passenger requests), recorded over a month's duration, in the area of Manhattan, New York. Based on current traffic statistics, our evaluations indicate that privacy can be achieved without incurring a significant loss of performance, and that this loss can be further controlled by varying operator or user preferences.

IROS Conference 2017 Conference Paper

Search-based motion planning for quadrotors using linear quadratic minimum time control

Sikang Liu 0002
Nikolay Atanasov 0001
Kartik Mohta
Vijay Kumar 0001

In this work, we propose a search-based planning method to compute dynamically feasible trajectories for a quadrotor flying in an obstacle-cluttered environment. Our approach searches for smooth, minimum-time trajectories by exploring the map using a set of short-duration motion primitives. The primitives are generated by solving an optimal control problem and induce a finite lattice discretization on the state space which can be explored using a graph-search algorithm. The proposed approach is able to generate resolution-complete (i. e. , optimal in the discretized space), safe, dynamically feasibility trajectories efficiently by exploiting the explicit solution of a Linear Quadratic Minimum Time problem. It does not assume a hovering initial condition and, hence, is suitable for fast online re-planning while the robot is moving. Quadrotor navigation with online re-planning is demonstrated using the proposed approach in simulation and physical experiments and comparisons with trajectory generation based on state-of-art quadratic programming are presented.

IROS Conference 2016 Conference Paper

A swarm of flying smartphones

Giuseppe Loianno
Yash Mulgaonkar
Chris Brunner
Dheeraj Ahuja
Arvind Ramanandan
Murali Chari
Serafin Diaz
Vijay Kumar 0001

In the last decade, consumer electronic devices such as smartphones, are packaged with small cameras, gyroscopes, and accelerometers, all sensors allowing autonomous deployment of aerial robots in GPS-denied environments. Our previous work [1], demonstrated the feasibility of using smartphones for autonomous flight. In many applications, there is a large interest to the use multiple autonomous aerial vehicles in a cooperative manner to speed up the operation of the mission. In this work, we present the first fully autonomous smartphone-based swarm of quadrotors. Multiple vehicles are able to plan safe trajectories avoiding inter-robot collisions, optimizing at the same time a given task and concurrently building in a cooperative manner a 3-D map of the environment. The sensing, sensor fusion, control, and planning are all done on an offthe- shelf Samsung Galaxy S5 smartphone using just the single camera and IMU available on the phone. The work allows any consumer with multiple smartphones to autonomously drive a swarm of multiple vehicles without GPS, by downloading an app, and have the swarm cooperatively map a 3-D environment.

ICRA Conference 2016 Conference Paper

Assembly sequence planning for constructing planar structures with rectangular modules

Jungwon Seo
Mark Yim
Vijay Kumar 0001

This paper addresses assembly sequence planning for constructing planar structures of the common brick wall pattern collectively with mobile modular robots that have the same rectangular footprint. Here we present a new algorithm for target structures with internal holes that our previous algorithm was not able to address. Our new algorithm constructs a feasible assembly sequence where robots do not have to pass through narrow corridors while approaching their target positions. The algorithm is provably correct and complete and runs in time that is linear in the size of a target structure, that is, the number of its parts. We also present software implementing our algorithms and a set of numerical experiments using the software. Finally, we extend our algorithms to address other symmetric patterns formed by a collection of congruent rectangles on the plane.

ICRA Conference 2016 Conference Paper

Control of microstructures propelled via bacterial baths

Elizabeth E. Hunter
Nathaniel Chodosh
Edward B. Steager
Vijay Kumar 0001

In this work, we explore a unique approach to harvest energy from active matter in order to independently power and control many untethered, mobile microrobots. More specifically, we investigate microstructure transport caused by stochastic collisions between freely swimming cells and rigid microstructure boundaries in a dense bacterial bath. We characterize the motion of asymmetrically shaped microstructures, specifically gears and chevrons, and demonstrate control of these microrobots by altering the motility of swimming cells with high intensity blue light. We demonstrate both rotational and translational motion control using a custom system for tracking and real-time automated exposure of local bath regions using visual feedback. These experiments are supported by a mathematical model and simulations which describe the dynamics of a microstructure propelled by many stochastic cellstructure collisions. This model allows us to predict microrobot trajectories and to develop improved strategies for microrobot design and control.

ICRA Conference 2016 Conference Paper

Formalizing the impact of diversity on performance in a heterogeneous swarm of robots

Amanda Prorok
M. Ani Hsieh
Vijay Kumar 0001

We are interested in a principled study of the impact of diversity in heterogeneous large-scale distributed robotic systems. In order to evaluate the implications of heterogeneity on performance, we consider the concrete problem of distributing a large group of robots among a set of tasks that require specialized capabilities in order to be completed. We model the system of heterogeneous robots as a community of species, where each species (robot type) is defined by the traits (capabilities) that it owns. We develop a continuous model of the system at a macroscopic level, and formulate an optimization problem that produces an optimal set of transition rates for each species, so that the desired trait distribution is reached as quickly as possible. In order to evaluate the effects of heterogeneity, we propose a diversity metric that defines the notion of eigenspecies. We show that our metric correlates with performance: the higher the cardinality of the eigenspecies, the harder it becomes to optimize the system. Our approach is validated over multiple levels of abstraction, and real robot results confirm its validity on physical platforms.

ECAI Conference 2016 Conference Paper

Hierarchical Strategy Synthesis for Pursuit-Evasion Problems

Rattanachai Ramaithitima
Siddharth Srivastava 0001
Subhrajit Bhattacharya
Alberto Speranzon
Vijay Kumar 0001

We present a novel approach for solving pursuit-evasion problems where multiple pursuers with limited sensing capabilities are used to detect all possible mobile evaders in a given environment. We make no assumptions about the number, the speed, or the maneuverability of evaders. Our algorithm takes as input a map of the environment and sensor models for the pursuers. We then obtain a graph representation of an environment using a &Ccaron; ech Complex. Even with such a representation, the configuration space grows exponentially with the number of pursuers. In order to address this challenge, we propose an abstraction framework to partition the configuration space into sets of topologically similar configurations that preserve the space of possible evader locations. We validate our approach on several simulated environments with varying topologies and numbers of pursuers.

ICRA Conference 2016 Conference Paper

High speed navigation for quadrotors with limited onboard sensing

Sikang Liu 0002
Michael Watterson
Sarah Y. Tang
Vijay Kumar 0001

We address the problem of high speed autonomous navigation of quadrotor micro aerial vehicles with limited onboard sensing and computation. In particular, we propose a dual range planning horizon method to safely and quickly navigate quadrotors to specified goal locations in previously unknown and unstructured environments. In each planning epoch, a short-range planner uses a local map to generate a new trajectory. At the same time, a safe stopping policy is found. This allows the robot to come to an emergency halt when necessary. Our algorithm guarantees collision avoidance and demonstrates important advances in real-time planning. First, our novel short range planning method allows us to generate and re-plan trajectories that are dynamically feasible, comply with state and input constraints, and avoid obstacles in real-time. Further, previous planning algorithms abstract away the obstacle detection problem by assuming the instantaneous availability of geometric information about the environment. In contrast, our method addresses the challenge of using the raw sensor data to form a map and navigate in real-time. Finally, in addition to simulation examples, we provide physical experiments that demonstrate the entire algorithmic pipeline from obstacle detection to trajectory execution.

ICRA Conference 2016 Conference Paper

Hybrid architecture for communication-aware multi-robot systems

James Stephan
Jonathan Fink
Vijay Kumar 0001
Alejandro Ribeiro

In this paper we propose a hybrid architecture that allows a team of mobile robots to self-organize into a multi-hop ad-hoc network and solve the joint mobility and communication problem in complex environments to complete a given task. The system consists of an outer global planning loop and an inner local loop responsible for motion and network routing, arranged in a two-stage feedback system. This system is able to leverage the benefits of previous systems, while avoiding their drawbacks. This results in a lightweight responsive system that is able to operate in complex environments with minimal global coordination while maintaining a minimum end-to-end data rate between robots. Two main benefits of our approach are demonstrated through experimentation superior performance over existing systems and dynamic adjustment to unexpected events. We conclude with a demonstration of the system operating in a realistic scenario, in which the team patrols a set of hallways.

IROS Conference 2016 Conference Paper

Online planning for energy-efficient and disturbance-aware UAV operations

Nicola Bezzo
Kartik Mohta
Cameron Nowzari
Insup Lee 0001
Vijay Kumar 0001
George J. Pappas

In this paper we consider an online planning problem for unmanned aerial vehicle (UAV) operations. Specifically, a UAV has the task of reaching a goal from a set of possible goals while minimizing the amount of energy required. Due to unforeseen disturbances, it is possible that initially attractive goals might end up being very expensive during the execution. Thus, two main problems are investigated here: i) how to predict and plan the motion of the UAV at run time to minimize its energy consumption and ii) when to schedule next replanning time to avoid unnecessary periodic re-evaluation executions. Our approach considers a nonlinear model of the system for which a model predictive controller is used to determine the desired control inputs for each possible goal. These control inputs are then used to estimate the energy required to reach the different goals. Finally, a self-triggered scheduling policy determines how long to wait before replanning the goal to aim for. The proposed framework is validated through simulations and experiments in which a quadrotor must choose and reach some goal while being subject to external disturbances.

IROS Conference 2016 Conference Paper

Safe and complete trajectory generation for robot teams with higher-order dynamics

Sarah Y. Tang
Vijay Kumar 0001

In this work, we consider the labeled multi-robot planning problem. In this paradigm, a team of robots at fixed start positions must navigate to pre-specified and noninterchangable goal positions. While many algorithms have been proposed for finding optimal solutions to this problem, most methods assume that the robots are kinematic agents, whereas in reality, robots often have high-order dynamics that must be respected by their trajectories. Here, we propose a centralized method for generating trajectories for teams of robots with general nth-order dynamics navigating to labeled goals. Our algorithm is safe and complete and additionally allows for decoupled optimization of each robot's trajectory as a Quadratic Program with linear constraints. We present simulation results for teams of up to 20 robots.

IROS Conference 2016 Conference Paper

Smooth trajectory generation on SE(3) for a free flying space robot

Michael Watterson
Trey Smith
Vijay Kumar 0001

We propose a new optimal trajectory generation technique on SE(3) which avoids known obstacles. We leverage techniques from differential geometry and Lie algebra to formulate a cost functional which is intrinsic to the geometric structure of this space and makes physical sense. We propose an approximation technique to generate trajectories on the subgroup SO(3) and use Semidefinite Programming (SDP) to approximate an NP-Hard problem with one which is tractable to compute. From this trajectory on the subgroup, the trajectory generation on the other dimensions of the group becomes a Quadratic Program (QP). For obstacle avoidance, we use a computational geometric technique to decompose the environment into overlapping convex regions to confine the trajectory. We show how this motion planning technique can be used to generate feasible trajectories for a space robot in SE(3) and describe controllers that enable the execution of the generated trajectory. We compare our method to other geometric techniques for calculating trajectories on SO(3) and SE(3), but in an obstacle-free environment.

ICRA Conference 2016 Conference Paper

The flying monkey: A mesoscale robot that can run, fly, and grasp

Yash Mulgaonkar
Brandon Araki
Je-Sung Koh
Luis Guerrero-Bonilla
Daniel M. Aukes
Anurag Makineni
Michael T. Tolley
Daniela Rus

The agility and ease of control make a quadrotor aircraft an attractive platform for studying swarm behavior, modeling, and control. The energetics of sustained flight for small aircraft, however, limit typical applications to only a few minutes. Adding payloads - and the mechanisms used to manipulate them - reduces this flight time even further. In this paper we present the flying monkey, a novel robot platform having three main capabilities: walking, grasping, and flight. This new robotic platform merges one of the world's smallest quadrotor aircraft with a lightweight, single-degree-of-freedom walking mechanism and an SMA-actuated gripper to enable all three functions in a 30 g package. The main goal and key contribution of this paper is to design and prototype the flying monkey that has increased mission life and capabilities through the combination of the functionalities of legged and aerial robots.

ICRA Conference 2016 Conference Paper

Towards autonomous phytopathology: Outcomes and challenges of citrus greening disease detection through close-range remote sensing

Suproteem K. Sarkar
Jnaneshwar Das
Reza Ehsani
Vijay Kumar 0001

Unmanned aerial vehicles (UAVs) have the potential to significantly impact early detection and monitoring of plant diseases. In this paper, we present preliminary work in developing a UAV-mounted sensor suite for detection of citrus greening disease, a major threat to Florida citrus production. We propose a depth-invariant sensing methodology for measuring reflectance of polarized amber light, a metric which has been found to measure starch accumulation in greening-infected leaves. We describe the implications of adding depth information to this method, including the use of machine learning models to discriminate between healthy and infected leaves with validation accuracies up to 93%. Additionally, we discuss stipulations and challenges of use of the system with UAV platforms. This sensing system has the potential to allow for rapid scanning of groves to determine the spread of the disease, especially in areas where infection is still in early stages, including citrus farms in California. Although presented in the context of citrus greening disease, the methods can be applied to a variety of plant pathology studies, enabling timely monitoring of plant health—impacting scientists, growers, and policymakers.

IROS Conference 2016 Conference Paper

Towards fully autonomous visual inspection of dark featureless dam penstocks using MAVs

Tolga Özaslan
Kartik Mohta
James Keller 0002
Yash Mulgaonkar
Camillo J. Taylor
Vijay Kumar 0001
Jennifer M. Wozencraft
Thomas Hood

In the last decade, multi-rotor Micro Aerial Vehicles (MAVs) have attracted great attention from robotics researchers. Offering affordable agility and maneuverability, multi-rotor aircrafts have become the most commonly used platforms for robotics applications. Amongst the most promising applications are inspection of power-lines, cell-towers, large and constrained infrastructures and precision agriculture. While GPS offers an easy solution for outdoor autonomy, using on-board sensors is the only solution for autonomy in constrained indoor environments. In this paper, we present our results on autonomous inspection of completely dark, featureless, symmetric dam penstocks using cameras and range sensors. We use a hex-rotor platform equipped with an IMU, four cameras and two lidars. One of the cameras tracks features on the walls using the on-board illumination to estimate the position along the tunnel axis unobservable to range sensors while all of the cameras are used for panoramic image construction. The two lidars estimate the remaining degrees of freedom (DOF). Outputs of the two estimators are fused using an Unscented Kalman Filter (UKF). A moderately trained operator defines waypoints using the Remote Control (RC). We demonstrate our results from Carters Dam, GA and Glen Canyon Dam, AZ which include panoramic images for cracks and rusty spot detection and 6-DOF estimation results with ground truth comparisons. To our knowledge ours is the only study that can autonomously inspect environments with no geometric cues and poor to no external illumination using MAVs.

ICRA Conference 2016 Conference Paper

Versatile aerial grasping using self-sealing suction

Chad C. Kessens
Justin Thomas
Jaydev P. Desai
Vijay Kumar 0001

This paper addresses the challenge of versatile aerial grasping utilizing suction while considering the limitations of an on-board vacuum pump. It builds upon our patented self-sealing suction cup technology, which allows the exertion of local pulling contact forces for grasping a wide range of objects. The novel self-sealing nature of the cups enables the gripper to be versatile, employing just one, several, or all of the cups for the grasp in a passively actuated manner. We begin by describing the design of the system and its components. Because aerial applications are typically sensitive to weight constraints, we used a micro-pump vacuum generator, which introduced new challenges for our system. To investigate and overcome those challenges, we tested the relationship between the cup's design and its leakage, activation force, and maximum holding force. In addition, we tested the performance of the individual gripper components, the aerial vehicle's ability to transfer force to the cups, the system's ability to grip inclined surfaces, and finally the vehicle's ability to grasp a multitude of objects using various numbers of cups. This included the grasping of one object, followed by the grasping of a second object while still holding the first object.

ICRA Conference 2016 Conference Paper

Visual inertial odometry for quadrotors on SE(3)

Giuseppe Loianno
Michael Watterson
Vijay Kumar 0001

The combination of on-board sensors measurements with different statistical characteristics can be employed in robotics for localization and control, especially in GPS-denied environments. In particular, most aerial vehicles are packaged with low cost sensors, important for aerial robotics, such as camera, a gyroscope, and an accelerometer. In this work, we develop a visual inertial odometry system based on the Unscented Kalman Filter (UKF) acting on the Lie group SE(3), such to obtain an unique, singularity-free representation of a rigid body pose. We model this pose with the Lie group SE(3) and model the noise on the corresponding Lie algebra. Moreover, we extend the concepts used in the standard UKF formulation, such as state uncertainty and modeling, to correctly incorporate elements that do not belong to an Euclidean space such as the Lie group members. In this analysis, we use the parallel transport, which requires us to explicitly consider SE(3) as representing rigid bodies though the use of the affine connection. We present experimental results to show the effectiveness of the proposed approach for state estimation of a quadrotor platform.

ICRA Conference 2016 Conference Paper

Visual-inertial direct SLAM

Alejo Concha
Giuseppe Loianno
Vijay Kumar 0001
Javier Civera 0001

The so-called direct visual SLAM methods have shown a great potential in estimating a semidense or fully dense reconstruction of the scene, in contrast to the sparse reconstructions of the traditional feature-based algorithms. In this paper, we propose for the first time a direct, tightly-coupled formulation for the combination of visual and inertial data. Our algorithm runs in real-time on a standard CPU. The processing is split in three threads. The first thread runs at frame rate and estimates the camera motion by a joint non-linear optimization from visual and inertial data given a semidense map. The second one creates a semidense map of high-gradient areas only for camera tracking purposes. Finally, the third thread estimates a fully dense reconstruction of the scene at a lower frame rate. We have evaluated our algorithm in several real sequences with ground truth trajectory data, showing a state-of-the-art performance.

ICRA Conference 2015 Conference Paper

Autonomous robotic exploration using occupancy grid maps and graph SLAM based on Shannon and Rényi Entropy

Henry Carrillo
Philip M. Dames
Vijay Kumar 0001
José A. Castellanos 0001

In this paper we examine the problem of autonomously exploring and mapping an environment using a mobile robot. The robot uses a graph-based SLAM system to perform mapping and represents the map as an occupancy grid. In this setting, the robot must trade-off between exploring new area to complete the task and exploiting the existing information to maintain good localization. Selecting actions that decrease the map uncertainty while not significantly increasing the robot's localization uncertainty is challenging. We present a novel information-theoretic utility function that uses both Shannon's and Rényi's definitions of entropy to jointly consider the uncertainty of the robot and the map. This allows us to fuse both uncertainties without the use of manual tuning. We present simulations and experiments comparing the proposed utility function to state-of-the-art utility functions, which only use Shannon's entropy. We show that by using the proposed utility function, the robot and map uncertainties are smaller than using other existing methods.

ICRA Conference 2015 Conference Paper

Comparison of an attitude estimator based on the Lagrange-d'Alembert principle with some state-of-the-art filters

Maziar Izadi
Ehsan Samiei
Amit K. Sanyal
Vijay Kumar 0001

Discrete-time estimation of rigid body attitude and angular velocity without any knowledge of the attitude dynamics model, is treated using the discrete Lagrange-d'Alembert principle. Using body-fixed sensor measurements of direction vectors and angular velocity, a Lagrangian is obtained as the difference between a kinetic energy-like term that is quadratic in the angular velocity estimation error, and an artificial potential obtained from Wahba's function. An additional dissipation term that depends linearly on the angular velocity estimation error is introduced, and the discrete Lagrange-d'Alembert principle is applied to the Lagrangian with this dissipation. An implicit and an explicit first-order version of this discrete-time estimation scheme is presented. A comparison of this estimator is made with certain state-of-the-art attitude estimators in the absence of bias in sensor readings. Numerical simulations show that this estimator is robust and unlike extended Kalman filter-based schemes, its convergence does not depend on the gain values. In addition, the variational estimator is found to be more computationally efficient than these other estimators.

ICRA Conference 2015 Conference Paper

Cooperative localization and mapping of MAVs using RGB-D sensors

Giuseppe Loianno
Justin Thomas
Vijay Kumar 0001

The fusion of IMU and RGB-D sensors presents an interesting combination of information to achieve autonomous localization and mapping using robotic platforms such as ground robots and flying vehicles. In this paper, we present a software framework for cooperative localization and mapping while simultaneously using multiple aerial platforms. We employ a monocular visual odometry algorithm to solve the localization task, where the depth data flow associated to the RGB image is used to estimate the scale factor associated with the visual information. The current framework enables autonomous onboard control of each vehicle with cooperative localization and mapping. We present a methodology that provides both a sparse map generated by the monocular SLAM and a multiple resolution dense map generated by the associated depth. The localization algorithm and both 3D mapping algorithms work in parallel improving the system real-time reliability. We present experimental results to show the effectiveness of the proposed approach using two quadrotors platforms.

ICRA Conference 2015 Conference Paper

Design of small, safe and robust quadrotor swarms

Yash Mulgaonkar
Gareth Cross
Vijay Kumar 0001

Scaling down the size and mass of micro aerial vehicles (MAVs) increases their agility and their ability to operate in tight formations. In addition, smaller robots are safer and, as we will show in this paper, more robust to collisions. This paper addresses the development of a pico quadrotor measuring 11 cm from tip to tip, with a mass of 25g. To increase the robustness of the robot to collisions, the vehicle is equipped with a 2 gram carbon fiber cage that protects it from impact velocities in excess of 4 m/s and also permits recovery after collisions. We present the design of the electrical, mechanical and computational elements, as well as experimental results demonstrating trajectory following with feedback from an external motion camera system, recovery from collisions with walls and other robots, and formation flight.

ICRA Conference 2015 Conference Paper

Information-theoretic mapping using Cauchy-Schwarz Quadratic Mutual Information

Benjamin Charrow
Sikang Liu 0002
Vijay Kumar 0001
Nathan Michael

We develop a computationally efficient control policy for active perception that incorporates explicit models of sensing and mobility to build 3D maps with ground and aerial robots. Like previous work, our policy maximizes an information-theoretic objective function between the discrete occupancy belief distribution (e. g. , voxel grid) and future measurements that can be made by mobile sensors. However, our work is unique in three ways. First, we show that by using Cauchy-Schwarz Quadratic Mutual Information (CSQMI), we get significant gains in efficiency. Second, while most previous methods adopt a myopic, gradient-following approach that yields poor convergence properties, our algorithm searches over a set of paths and is less susceptible to local minima. In doing so, we explicitly incorporate models of sensors, and model the dependence (and independence) of measurements over multiple time steps in a path. Third, because we consider models of sensing and mobility, our method naturally applies to both ground and aerial vehicles. The paper describes the basic models, the problem formulation and the algorithm, and demonstrates applications via simulation and experimentation.

ICRA Conference 2015 Conference Paper

Mixed Integer Quadratic Program trajectory generation for a quadrotor with a cable-suspended payload

Sarah Y. Tang
Vijay Kumar 0001

In this paper, we present a trajectory planning method to navigate a quadrotor with a cable-suspended payload through known obstacle-filled environments. We model the system as a hybrid dynamical system and formulate the trajectory generation problem as a Mixed Integer Quadratic Program (MIQP). Specifically, we address two novel challenges. First, we plan for a multi-body system, and obstacle avoidance must be guaranteed for the quadrotor, load, and the cable. Second, our method accommodates transitions between subsystems of the hybrid dynamical system, allowing for maneuvers that would otherwise be infeasible if the cable were constrained to remain taut. Numerical and experimental results validate the proposed approach for the full hybrid system.

ICRA Conference 2015 Conference Paper

Optically addressing microscopic bioactuators for real-time control

Edward B. Steager
Denise Wong
Nathaniel Chodosh
Vijay Kumar 0001

Independent control of several, untethered microrobots remains a significant challenge. Although progress has been made in controlling multiple robots at scales below 100 mm, motion is dependent on external energy sources for propulsion and is often not fully decoupled. Here, we develop a method to dynamically pattern light at a resolution of 1 µm 2 per pixel in a closed loop real-time system using a digital micromirror device (DMD) system. We test the system on microbiorobots - bacteria-coated microplates - and demonstrate that we can selectively stimulate pre-defined regions of bioactuators. We verify previously developed dynamic models as a step toward developing individually addressable, self-propelled microrobots.

IROS Conference 2015 Conference Paper

Safe receding horizon control for aggressive MAV flight with limited range sensing

Michael Watterson
Vijay Kumar 0001

Micro Aerial Vehicles (MAVs) are becoming ubiquitous, but most experiments and demonstrations have been limited to slow flight except in open environments or in laboratories with motion capture systems. In this paper, we develop representations and algorithms for aggressive flight in cluttered environments. We incorporate the specific dynamics of the system and generate safe, feasible trajectories for fast navigation in real time. Specifically, we use a polyhedral decomposition of the visible free space and address the generation of safe trajectories that are within the space. Because of the limited field of view, we adopt a receding horizon control policy (RHCP) for planning over a finite time horizon, but with the guarantee that there exists a safe stopping control policy over a second time horizon from the planned state at the end of the horizon. Thus, the robot planning occurs over two horizons. While the robot executes the planned trajectory over the first time horizon, the map of obstacles is refreshed allowing the planner to generate a refined plan, once again over two time horizons. The key algorithmic contribution of the paper lies in the fast planning algorithm that is able to incorporate robot dynamics while guaranteeing safety. The algorithm is also optimal in the sense that the receding horizon control policy is based on minimizing the trajectory snap. Central to the algorithm is a novel polyhedral representation that allows us to abstract the trajectory planning problem as a problem of finding a path through a sequence of convex regions in configuration space.

ICRA Conference 2015 Conference Paper

Sensor coverage robot swarms using local sensing without metric information

Rattanachai Ramaithitima
Michael Whitzer
Subhrajit Bhattacharya
Vijay Kumar 0001

We consider the problem of deploying a swarm of mobile robots into an unknown environment for attaining complete sensor coverage of the environment. The robots have limited and noisy sensing capabilities and no metric or global information available to them. Using tools from algebraic topology, we formally describe the sensor coverage as a simplicial complex, deploy robots through the complex using bearing-based local controllers, and attain coverage while identifying and removing sensor redundancies. Despite the highly limited sensing capabilities and complete lack of global localization and metric information, we demonstrate that the proposed algorithm is complete, always terminates in a finite-sized environment, is guaranteed to attain complete coverage and is robust to sensor failures. The algorithm presented in this paper was demonstrated through simulation and proves to effectively cover and explore unknown indoor environments.

IROS Conference 2015 Conference Paper

Smartphones power flying robots

Giuseppe Loianno
Yash Mulgaonkar
Chris Brunner
Dheeraj Ahuja
Arvind Ramanandan
Murali Chari
Serafin Diaz
Vijay Kumar 0001

Consumer grade technology seen in cameras and phones has led to the price/performance ratio of sensors and processors falling dramatically over the last decade. In particular, most devices are packaged with a camera, a gyroscope, and an accelerometer, important sensors for aerial robotics. The low mass and small form factor make them particularly well suited for autonomous flight with small flying robots, especially in GPS-denied environments. In this work, we present the first fully autonomous smartphone-based quadrotor. All the computation, sensing and control runs on an off-the-shelf smartphone, with all the software functionality in a smartphone app. We show how quadrotors can be stabilized and controlled to achieve autonomous flight in indoor buildings with application to smart homes, search and rescue, construction and architecture. The work allows any consumer with a smartphone to autonomously drive a quadrotor robot platform, even without GPS, by downloading an app, and concurrently build 3-D maps.

ICRA Conference 2015 Conference Paper

Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs

Shaojie Shen
Nathan Michael
Vijay Kumar 0001

There have been increasing interests in the robotics community in building smaller and more agile autonomous micro aerial vehicles (MAVs). In particular, the monocular visual-inertial system (VINS) that consists of only a camera and an inertial measurement unit (IMU) forms a great minimum sensor suite due to its superior size, weight, and power (SWaP) characteristics. In this paper, we present a tightly-coupled nonlinear optimization-based monocular VINS estimator for autonomous rotorcraft MAVs. Our estimator allows the MAV to execute trajectories at 2 m/s with roll and pitch angles up to 30 degrees. We present extensive statistical analysis to verify the performance of our approach in different environments with varying flight speeds.

IROS Conference 2015 Conference Paper

Visibility-based persistent monitoring with robot teams

Pratap Tokekar
Vijay Kumar 0001

We study the problem of planning paths for a team of robots motivated by coverage, persistent monitoring and surveillance applications. The input is a set of target points in a polygonal environment that must be monitored using robots with omni-directional cameras. The goal is to compute paths for all robots such that every target is visible from at least one path. The cost of a path is given by the weighted combination of the length of the path (travel time) and the number of viewpoints along the path (measurement time). The overall cost is given by the maximum cost over all robot paths and the objective is to minimize the maximum cost. In its general form, this problem is NP-hard. In this paper, we present an optimal algorithm and a constant factor approximation for two special versions of the problem. In both cases, the paths are restricted to lie on a pre-defined curve in the polygon. We show that if the curve satisfies a special property, termed chain-visibility, then there exists an optimal algorithm for monitoring a given set of target locations. Furthermore, if we restrict the input polygon to the class of street polygons, then we present a constant-factor approximation which is applicable even if the set of target locations is the entire polygon. In addition to theoretical proofs, we also present results from simulation studies.

IROS Conference 2014 Conference Paper

Automated composition of motion primitives for multi-robot systems from safe LTL specifications

Indranil Saha 0001
Rattanachai Ramaithitima
Vijay Kumar 0001
George J. Pappas
Sanjit A. Seshia

We present a compositional motion planning framework for multi-robot systems based on an encoding to satisfiability modulo theories (SMT). In our framework, the desired behavior of a group of robots is specified using a set of safe linear temporal logic (LTL) properties. Our method relies on a library of motion primitives, each of which corresponds to a controller that ensures a particular trajectory in a given configuration. Using the closed-loop behavior of the robots under the action of different controllers, we formulate the motion planning problem as an SMT solving problem and use an off-the-shelf SMT solver to generate trajectories for the robots. Our approach can also be extended to synthesize optimal cost trajectories where optimality is defined with respect to the available motion primitives. Experimental results show that our framework can efficiently solve complex motion planning problems in the context of multi-robot systems.

ICRA Conference 2014 Conference Paper

Decentralized goal assignment and trajectory generation in multi-robot networks: A multiple Lyapunov functions approach

Dimitra Panagou
Matthew Turpin
Vijay Kumar 0001

This paper considers the problem of decentralized goal assignment and trajectory generation for multi-robot networks when only local communication is available, and proposes an approach based on methods related to switched systems and set invariance. A family of Lyapunov-like functions is employed to encode the (local) decision making among candidate goal assignments, under which the agents pick the assignment which results in the shortest total distance to the goals. An additional family of Lyapunov-like barrier functions is activated in the case when the optimal assignment may lead to colliding trajectories, thus maintaining system safety while preserving the convergence guarantees. The proposed switching strategies give rise to feedback control policies which are scalable as the number of agents increases, and therefore are suitable for applications including first-response deployment of robotic networks under limited information sharing. Simulations demonstrate the efficacy of the proposed method.

ICRA Conference 2014 Conference Paper

Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft MAV

Shaojie Shen
Yash Mulgaonkar
Nathan Michael
Vijay Kumar 0001

We present a modular and extensible approach to integrate noisy measurements from multiple heterogeneous sensors that yield either absolute or relative observations at different and varying time intervals, and to provide smooth and globally consistent estimates of position in real time for autonomous flight. We describe the development of algorithms and software architecture for a new 1. 9kg MAV platform equipped with an IMU, laser scanner, stereo cameras, pressure altimeter, magnetometer, and a GPS receiver, in which the state estimation and control are performed onboard on an Intel NUC 3 rd generation i3 processor. We illustrate the robustness of our framework in large-scale, indoor-outdoor autonomous aerial navigation experiments involving traversals of over 440 meters at average speeds of 1. 5 m/s with winds around 10 mph while entering and exiting buildings.

ICRA Conference 2014 Conference Paper

Path planning for a tethered mobile robot

Soonkyum Kim
Subhrajit Bhattacharya
Vijay Kumar 0001

In this paper we investigate the problem of navigation for a planar mobile robot tethered to a base by a flexible cable of length L. Obstacles present in the environment, coupled with the cable length constraint, makes the problem highly non-trivial. We adopt a topological approach along with graph search-based techniques to solve this problem, wherein we use the notion of a homotopy augmented graph to capture the information about the homotopy class of the cable. This lets us plan traversable optimal trajectories from an initial robot position and cable configuration to a final position of the robot. We demonstrate the algorithm by planning trajectories in several cluttered environments and with different cable lengths. As a demonstration of practical applicability, using a dynamic simulation testbed we simulate a robot-cable system following a planned trajectory.

IROS Conference 2014 Conference Paper

RF odometry for localization in pipes based on periodic signal fadings

Carlos Rizzo
Vijay Kumar 0001
Francisco Lera
José Luis Villarroel

Accurate localization is a problem in environments such as tunnels or pipes due to the hostile conditions, dimensions and the general lack of distinctive visual and/or structural features. Standard indoor localization techniques (e. g. visual SLAM) do not work well in pipelines given the lack of exploitable visual features, while outdoor techniques (GPS in particular) do not work inside metal pipes. In this paper, we present a RF odometry-like method to localize a robot along a pipe. Using a radio-frequency signal transmitter and a receiver, we propose and implement the complete setup in order to obtain periodic received-signal fadings and base a localization system on the periodicity of these fadings. There are two main advantages of the proposed system. First, the sensors are easy to install and can be used with any (ground, aerial) robot. Second, the fadings obtained are periodic, avoiding cumulative errors in localization.

IROS Conference 2014 Conference Paper

Robust routing and Multi-Confirmation Transmission Protocol for connectivity management of mobile robotic teams

James Stephan
Jonathan Fink
Benjamin Charrow
Alejandro Ribeiro
Vijay Kumar 0001

Providing reliable end-to-end communication for teams of robots requires the integration of novel routing techniques, motion planning algorithms, and transport level communication protocols. In this paper we look at existing robust routing solutions that provide redundancy at the routing layer and develop the Multi-Confirmation Transmission Protocol (MCTP) to take advantage of that redundancy at the transport level. The resulting system that integrates robust routing and MCTP is evaluated in experiments performed in complex environments. The integrated system is observed to provide a robust architecture that allows for near lossless communication while operating in a complex environment with less traffic than standard confirmation protocols.

ICRA Conference 2014 Conference Paper

Self-assembly of a swarm of autonomous boats into floating structures

Ian O'Hara
James Paulos
Jay Davey
Nick Eckenstein
Neel Doshi
Tarik Tosun
Jonathan Greco
Jungwon Seo

This paper addresses the self-assembly of a large team of autonomous boats into floating platforms. We describe the design of individual boats, the systems concept, the algorithms, the software architecture and experimental results with prototypes that are 1: 12 scale realizations of modified ISO shipping containers, with the goal of demonstrating self-assembly into large maritime structures such as air strips, bridges, harbors or sea bases. Each container is a robotic module capable of holonomic motion that can dock in a brick pattern to form arbitrary shapes. Over 60 modules were built of varying capability. The docking mechanism is designed to be robust to large disturbances that can be expected in the high seas. The docking mechanism also incorporates adjustable stiffness so that the conglomerate can comply to waves representative of sea state three, and have the ability to dynamically stiffen as required. The component modules for autonomous assembly, docking and simultaneous collision-free planning as well as the software architecture are presented along with the description of experimental verification.

ICRA Conference 2014 Conference Paper

Sensors for micro bio robots via synthetic biology

Edward B. Steager
Denise Wong
Deepak Mishra
Ron Weiss
Vijay Kumar 0001

Microscale robots offer an unprecedented opportunity to perform tasks at resolutions approaching 1 μm, but the great majority of research to this point focuses on actuation and control. Potential applications for microrobots can be considerably expanded by integrating sensing, signal processing and feedback into the system. In this work, we demonstrate that technologies from the field of synthetic biology may be directly integrated into microrobotic systems to create cell-based programmable mobile sensors, with signal processors and memory units. Specifically, we integrate genetically engineered, ultraviolet light-sensing bacteria with magnetic microrobots, creating the first controllable biological microrobot that is capable of exploring, recording and reporting on the state of the microscale environment. We demonstrate two proof-of-concept prototypes: (a) an integrated microrobot platform that is able to sense biochemical signals, and (b) a microrobot platform that is able to deploy biosensor payloads to monitor biochemical signals, both in a biological environment. These results have important implications for integrated micro-bio-robotic systems for applications in biological engineering and research.

ICRA Conference 2014 Conference Paper

Toward image based visual servoing for aerial grasping and perching

Justin Thomas
Giuseppe Loianno
Koushil Sreenath
Vijay Kumar 0001

This paper addresses the dynamics, control, planning, and visual servoing for micro aerial vehicles to perform high-speed aerial grasping tasks. We draw inspiration from agile, fast-moving birds, such as raptors, that detect, locate, and execute high-speed swoop maneuvers to capture prey. Since these grasping maneuvers are predominantly in the sagittal plane, we consider the planar system and present mathematical models and algorithms for motion planning and control, required to incorporate similar capabilities in quadrotors equipped with a monocular camera. In particular, we develop a dynamical model directly in the image space, show that this is a differentially-flat system with the image features serving as flat outputs, outline a method for generating trajectories directly in the image feature space, develop a geometric visual controller that considers the second order dynamics (in contrast to most visual servoing controllers that assume first order dynamics), and present validation of our methods through both simulations and experiments.

ICRA Conference 2014 Conference Paper

Vision-based control of a quadrotor for perching on lines

Kartik Mohta
Vijay Kumar 0001
Kostas Daniilidis

We formulate the position-based visual servoing problem for a quadrotor equipped with a monocular camera and an IMU relying only on features on planes and lines in order to fly above and perch on arbitrarily oriented lines. We show that we are able to compute the orientation of an arbitrarily oriented line, the speed of the robot and its position with respect to the target line using two points at a known distance on the line. The direction of the velocity is derived from optical flow induced by features on a plane in the background Finally, we demonstrate fully autonomous flight and perching using a small 230 gram quadrotor with all the computations running on the robot.

ICRA Conference 2013 Conference Paper

A single planner for a composite task of approaching, opening and navigating through non-spring and spring-loaded doors

Steven Gray 0003
Sachin Chitta
Vijay Kumar 0001
Maxim Likhachev

Opening and navigating through doors remains a challenging problem, particularly in cluttered environments and for spring-loaded doors. Passing through doors, especially spring-loaded doors, requires making and breaking contacts with the door and preventing the door from closing while passing through. In this work, we present a planning framework that handles non-spring and spring-loaded doors, in cluttered or confined workspaces, planning the approach to the door, pushing or pulling it open, and passing through. Because the problem is solved in a combined search space, the planner yields an overall least-cost path. The planner is able to insert a transition between robot-door contacts at any point along the plan. We utilize a compact graph-based representation of the problem to keep planning times low. We precompute the force workspace of the end-effectors to eliminate checks against joint torque limits at plan time. We have validated our solution in both simulation and real-world experiments on the PR2 mobile manipulation platform; the robot is able to successfully open a variety of spring-loaded and non-spring-loaded doors by pushing and pulling.

ICRA Conference 2013 Conference Paper

Assessment of protein binding with magnetic microrobots in fluid

Edward B. Steager
Blaine Zern
Mahmut Selman Sakar
Vladimir Muzykantov
Vijay Kumar 0001

For applications such as targeting therapeutic nanocarriers to specific tissues, as well as for the more general problem of assessing binding affinity between proteins, it is important to discover tools which may be used for both micromanipulation as well as force microscopy. In this work, we measure picoscale forces using microscale robots, using antibody-coated microbead probes as an interface. We develop a model for a detailed understanding of the relationship between robot velocity and applied force, and apply this to the measurement of binding forces between antibody-coated beads and protein-coated glass slides. We also verify binding between specific antibody/protein combinations relative to untargeted control combinations, and we measure the rate at which this binding occurs. This offers a truly novel technique for measuring binding forces between proteins.

ICRA Conference 2013 Conference Paper

Concurrent assignment and planning of trajectories for large teams of interchangeable robots

Matthew Turpin
Nathan Michael
Vijay Kumar 0001

This paper considers the problem of finding optimal time parameterized trajectories for N unlabeled robots navigating through a cluttered environment to N unlabeled goal locations where success is defined as every goal being reached by any robot. We propose a complete computationally-tractable algorithm for simultaneously finding trajectories and assignment of goal locations. This method is then demonstrated to have an upper complexity bound of that scales polynomially in the number of robots, O(N 3 ). The trajectories generated are guaranteed to be minimum length and collision free, while the assignment policy minimizes the maximum distance travelled. The key idea in the paper comes from the coupling between the optimal assignment, the properties of the resulting paths, and the set of valid priority assignments to the robots. These benefits result from structure in the solution to the optimal assignment to create a partial ordering of the robots, which in turn allows safe trajectories to be easily generated. Finally, we demonstrate the performance of the algorithm through simulations with tens and hundreds of robots operating in cluttered and confined environments.

ICRA Conference 2013 Conference Paper

Cooperative multi-target localization with noisy sensors

Philip M. Dames
Vijay Kumar 0001

This paper addresses the task of searching for an unknown number of static targets within a known obstacle map using a team of mobile robots equipped with noisy, limited field-of-view sensors. Such sensors may fail to detect a subset of the visible targets or return false positive detections. These measurement sets are used to localize the targets using the Probability Hypothesis Density, or PHD, filter. Robots communicate with each other on a local peer-to-peer basis and with a server or the cloud via access points, exchanging measurements and poses to update their belief about the targets and plan future actions. The server provides a mechanism to collect and synthesize information from all robots and to share the global, albeit time-delayed, belief state to robots near access points. We design a decentralized control scheme that exploits this communication architecture and the PHD representation of the belief state. Specifically, robots move to maximize mutual information between the target set and measurements, both self-collected and those available by accessing the server, balancing local exploration with sharing knowledge across the team. Furthermore, robots coordinate their actions with other robots exploring the same local region of the environment.

ICRA Conference 2013 Conference Paper

Incremental micro-UAV motion replanning for exploring unknown environments

Mihail Pivtoraiko
Daniel Mellinger
Vijay Kumar 0001

This paper describes an approach to motion generation for quadrotor micro-UAV's navigating cluttered and partially known environments. We pursue a graph search method that, despite the high dimensionality of the problem, the complex dynamics of the system and the continuously changing environment model is capable of generating dynamically feasible motions in real-time. This is enabled by leveraging the differential flatness property of the system and by developing a structured search space based on state lattice motion primitives. We suggest a greedy algorithm to generate these primitives off-line automatically, given the robot's motion model. The process samples the reachability of the system and reduces it to a set of representative, canonical motions that are compatible with the state lattice structure, which guarantees that any incremental replanning algorithm is able to produce smooth dynamically feasible motion plans while reusing previous computation between replans. Simulated and physical experimental results demonstrate real-time replanning due to the inevitable and frequent world model updates during micro-UAV motion in partially known environments.

IROS Conference 2013 Conference Paper

Planning for opportunistic surveillance with multiple robots

Dinesh Thakur
Maxim Likhachev
James Keller 0002
Vijay Kumar 0001
Vladimir N. Dobrokhodov
Kevin D. Jones
Jeff Wurz
Isaac Kaminer

We are interested in the multiple robot surveillance problem where robots must allocate waypoints to be visited among themselves and plan paths through different waypoints while avoiding obstacles. Furthermore, the robots are allocated specific times to reach their respective goal locations and as a result they have to decide which robots have to visit which waypoints. Such a problem has the challenge of computing the allocation of waypoints across robots, ordering for these waypoints and dynamical feasibility of the paths between waypoints. We present an algorithm that runs a series of graph searches to solve the problem and provide theoretical analysis that our approach yields an optimal solution. We present simulated results as well as experiments on two UAVs that validate the capability of our algorithm. For a single robot, we can solve instances having 10–15 waypoints and for multiple robots, instances having five robots and 10 waypoints can be solved.

IROS Conference 2013 Conference Paper

Topological exploration of unknown and partially known environments

Soonkyum Kim
Subhrajit Bhattacharya
Robert Ghrist
Vijay Kumar 0001

We present the mathematical framework and algorithms for multi-robot topological exploration of unknown environments in which the main goal is to identify the different topological classes of trajectories and thus efficiently distribute the task of exploration among different groups of robots. We consider two-dimensional configuration spaces. At any point in time, the robots' map consists of known, partially-mapped obstacles. The unknown, yet-to-be-explored area is mapped to a single point, thus giving us a quotient space. The topological classes on the quotient space allows us to define topological classes of trajectories connecting a robot pose to the unknown region in the original configuration space. Robots explore this configuration space choosing different homology classes when confronted by obstacles or walls. We illustrate the basic idea with simulations of small teams of robots. Experiments with a single robot illustrate the applicability of the method to robots that have small sensor footprints and limited computational resources. We also provide comparisons with a standard frontier-based algorithm.

ICRA Conference 2013 Conference Paper

Trajectory generation and control of a quadrotor with a cable-suspended load - A differentially-flat hybrid system

Koushil Sreenath
Nathan Michael
Vijay Kumar 0001

A quadrotor with a cable-suspended load with eight degrees of freedom and four degrees underactuation is considered and the system is established to be a differentially-flat hybrid system. Using the flatness property, a trajectory generation method is presented that enables finding nominal trajectories with various constraints that not only result in minimal load swing if required, but can also cause a large swing in the load for dynamically agile motions. A control design is presented for the system specialized to the planar case, that enables tracking of either the quadrotor attitude, the load attitude or the position of the load. Stability proofs for the controller design and experimental validation of the proposed controller are presented.

ICRA Conference 2013 Conference Paper

Vision-based state estimation for autonomous rotorcraft MAVs in complex environments

Shaojie Shen
Yash Mulgaonkar
Nathan Michael
Vijay Kumar 0001

In this paper, we consider the development of a rotorcraft micro aerial vehicle (MAV) system capable of vision-based state estimation in complex environments. We pursue a systems solution for the hardware and software to enable autonomous flight with a small rotorcraft in complex indoor and outdoor environments using only onboard vision and inertial sensors. As rotorcrafts frequently operate in hover or nearhover conditions, we propose a vision-based state estimation approach that does not drift when the vehicle remains stationary. The vision-based estimation approach combines the advantages of monocular vision (range, faster processing) with that of stereo vision (availability of scale and depth information), while overcoming several disadvantages of both. Specifically, our system relies on fisheye camera images at 25 Hz and imagery from a second camera at a much lower frequency for metric scale initialization and failure recovery. This estimate is fused with IMU information to yield state estimates at 100 Hz for feedback control. We show indoor experimental results with performance benchmarking and illustrate the autonomous operation of the system in challenging indoor and outdoor environments.

ICRA Conference 2012 Conference Paper

Automated biomanipulation of single cells

Edward B. Steager
Mahmut Selman Sakar
Ceridwen Magee
Monroe Kennedy III
Anthony Cowley
Vijay Kumar 0001

Transport of individual cells or chemical payloads on a subcellular scale is an enabling tool for the study of cellular communication, cell migration, and other localized phenomena. We present a magnetically actuated robotic system for the fully automated manipulation of cells and microbeads. Our strategy uses autofluorescent robotic transporters and fluorescently labeled microbeads to aid tracking and control in optically obstructed environments. We demonstrate automated delivery of microbeads infused with chemicals to specified positions on neurons.

ICRA Conference 2012 Conference Paper

Autonomous indoor 3D exploration with a micro-aerial vehicle

Shaojie Shen
Nathan Michael
Vijay Kumar 0001

In this paper, we propose a stochastic differential equation-based exploration algorithm to enable exploration in three-dimensional indoor environments with a payload constrained micro-aerial vehicle (MAV). We are able to address computation, memory, and sensor limitations by considering only the known occupied space in the current map. We determine regions for further exploration based on the evolution of a stochastic differential equation that simulates the expansion of a system of particles with Newtonian dynamics. The regions of most significant particle expansion correlate to unexplored space. After identifying and processing these regions, the autonomous MAV navigates to these locations to enable fully autonomous exploration. The performance of the approach is demonstrated through numerical simulations and experimental results in single and multi-floor indoor experiments.

ICRA Conference 2012 Conference Paper

Decentralized formation control with variable shapes for aerial robots

Matthew Turpin
Nathan Michael
Vijay Kumar 0001

We address formation control for a team of quadrotors in which the robots follow a specified group trajectory while safely changing the shape of the formation according to specifications. The formation is prescribed by shape vectors which dictate the relative separations and bearings between the robots, while the group trajectory is specified as the desired trajectory of a leader or a virtual robot in the group. Each robot plans its trajectory independently based on its local information of neighboring robots which includes both the neighbor's planned trajectory and an estimate of its state. We show that the decentralized trajectory planners (a) result in consensus on the planned trajectory for predefined shapes and (b) achieve safe reconfiguration when changing shapes.

ICRA Conference 2012 Conference Paper

Maintaining visibility for leader-follower formations in obstacle environments

Dimitra Panagou
Vijay Kumar 0001

This paper addresses the problem of controlling a leader-follower (L - F) formation of two unicycle mobile robots moving under visibility constraints in a known obstacle environment. Visibility constraints are realized as inequality state constraints that determine a visibility set K. Maintaining visibility is translated into controlling the robots so that system trajectories starting in K always remain in K. We provide the conditions under which visibility is maintained, as well as a feedback control scheme that forces F to converge and remain into a set of desired configurations w. r. t. L while maintaining visibility. We also propose a cooperative control scheme for the motion of the formation in a known obstacle environment, so that both collision avoidance and maintaining visibility are ensured. The proposed control schemes are decentralized, in the sense that there is no direct communication between the robots. The efficacy of our algorithms is evaluated through simulations.

ICRA Conference 2012 Conference Paper

Mixed-integer quadratic program trajectory generation for heterogeneous quadrotor teams

Daniel Mellinger
Aleksandr Kushleyev
Vijay Kumar 0001

We present an algorithm for the generation of optimal trajectories for teams of heterogeneous quadrotors in three-dimensional environments with obstacles. We formulate the problem using mixed-integer quadratic programs (MIQPs) where the integer constraints are used to enforce collision avoidance. The method allows for different sizes, capabilities, and varying dynamic effects between different quadrotors. Experimental results illustrate the method applied to teams of up to four quadrotors ranging from 65 to 962 grams and 21 to 67 cm in width following trajectories in three-dimensional environments with obstacles with accelerations approaching 1g.

ICRA Conference 2012 Conference Paper

Motion planning for robust wireless networking

Jonathan Fink
Alejandro Ribeiro
Vijay Kumar 0001

We propose an architecture and algorithms for maintaining end-to-end network connectivity for autonomous teams of robots. By adopting stochastic models of point-to-point wireless communication and computing robust solutions to the network routing problem, we ensure reliable connectivity during robot movement in complex environments. We fully integrate the solution to network routing with the choice of node positions through the use of randomized motion planning techniques. Experiments demonstrate that our method succeeds in navigating a complex environment while ensuring that end-to-end communication rates meet or exceed prescribed values within a target failure tolerance.

ICRA Conference 2012 Conference Paper

Planar, bimanual, whole-arm grasping

Jungwon Seo
Soonkyum Kim
Vijay Kumar 0001

We address the problem of synthesizing planar, bimanual, whole-arm grasps by developing the abstraction of an open chain gripper, an open, planar chain of rigid links and revolute joints contacting a planar, polygonal object, and introducing the concept of a generalized contact. Since two generalized contacts suffice for planar grasps, we leverage previous work on caging and immobilization for two contact grasps to construct an algorithm which synthesizes contact configurations for stable grasping. Simulations show that our methodology can be applied to grasp a wide range of planar objects without relying on special-purpose end-effectors. Representative experiments with the PR2 humanoid robot illustrate that this approach is practical.

IROS Conference 2012 Conference Paper

Spatial, bimanual, whole-arm grasping

Jungwon Seo
Vijay Kumar 0001

We address the problem of synthesizing spatial, bimanual, whole-arm grasps by developing the abstractions of an open chain gripper, an open, spatial chain of rigid links and joints between the links, contacting a polyhedral object, and a generalized contact. We show that every general polyhedron can be immobilized by three generalized contacts. We leverage previous work on immobilization to construct an algorithm that synthesizes contact configurations for stable grasping. Our methodology can be applied to grasp a wide range of objects without relying on special-purpose end-effectors as shown in simulations and experiments with a PR2 humanoid robot.

ICRA Conference 2011 Conference Paper

Autonomous multi-floor indoor navigation with a computationally constrained MAV

Shaojie Shen
Nathan Michael
Vijay Kumar 0001

In this paper, we consider the problem of autonomous navigation with a micro aerial vehicle (MAV) in indoor environments. In particular, we are interested in autonomous navigation in buildings with multiple floors. To ensure that the robot is fully autonomous, we require all computation to occur on the robot without need for external infrastructure, communication, or human interaction beyond high-level commands. Therefore, we pursue a system design and methodology that enables autonomous navigation with real-time performance on a mobile processor using only onboard sensors. Specifically, we address multi-floor mapping with loop closure, localization, planning, and autonomous control, including adaptation to aerodynamic effects during traversal through spaces with low vertical clearance or strong external disturbances. We present experimental results with ground truth comparisons and performance analysis.

ICRA Conference 2011 Conference Paper

Autonomous multi-floor indoor navigation with a computationally constrained micro aerial vehicle

Shaojie Shen
Nathan Michael
Vijay Kumar 0001

We are interested in the problem of surveilling and exploring environments that include both indoor and outdoor settings. Aerial vehicles offer mobility and perspective advantages over ground platforms and micro aerial vehicles (MAVs) are particularly applicable to buildings with multiple floors where stairwells can be an obstacle to ground vehicles. A challenge when operating in indoor environments is the lack of an external source of localization such as GPS. For these reasons, in this work we focus on autonomous navigation in buildings with multiple floors without requiring an external source of localization or prior knowledge of the environment. To ensure that the robot is fully autonomous, we require all computation to occur on the robot without need for external infrastructure, communication, or human interaction beyond high-level commands. Therefore, we pursue a system design and methodology capable of autonomous navigation with real-time performance on a mobile processor using only onboard sensors (Fig. 1); where in this work autonomous navigation considers multi-floor mapping with loop closure, localization, planning, and control.

ICRA Conference 2011 Conference Paper

Cooperative control of autonomous surface vehicles for oil skimming and cleanup

Subhrajit Bhattacharya
Hordur Kristinn Heidarsson
Gaurav S. Sukhatme
Vijay Kumar 0001

Oil skimmers towed by two vehicles have been widely used for skimming of oil on the water surface. In this paper, we address the cooperative control of two autonomous surface vehicles for oil skimming and cleanings. We model the skimmer as a flexible, floating rope of constant length as well as discrete segmented model. We derive the equations governing the rope dynamics from first principles and demonstrate their application through simulations. We have performed field experiments with two autonomous surface vehicles that substantiate the proposed model and provides estimates of constants underlying the model. We propose a method for controlling the shape of the rope, and derive the conditions that maximize skimming efficiency.

ICRA Conference 2011 Conference Paper

Design of control policies for spatially inhomogeneous robot swarms with application to commercial pollination

Spring Berman
Vijay Kumar 0001
Radhika Nagpal

We present an approach to designing scalable, decentralized control policies that produce a desired collective behavior in a spatially inhomogeneous robotic swarm that emulates a system of chemically reacting molecules. Our approach is based on abstracting the swarm to an advection-diffusion-reaction partial differential equation model, which we solve numerically using smoothed particle hydrodynamics (SPH), a meshfree technique that is suitable for advection-dominated systems. The parameters of the macroscopic model are mapped onto the deterministic and random components of individual robot motion and the probabilities that determine stochastic robot task transitions. For very large swarms that are prohibitively expensive to simulate, the macroscopic model, which is independent of the population size, is a useful tool for synthesizing robot control policies with guarantees on performance in a top-down fashion. We illustrate our methodology by formulating a model of rabbiteye blueberry pollination by a swarm of robotic bees and using the macroscopic model to select control policies for efficient pollination.

IROS Conference 2011 Conference Paper

Design, modeling, estimation and control for aerial grasping and manipulation

Daniel Mellinger
Quentin Lindsey
Michael Shomin
Vijay Kumar 0001

This paper addresses mechanics, design, estimation and control for aerial grasping. We present the design of several light-weight, low-complexity grippers that allow quadrotors to grasp and perch on branches or beams and pick up and transport payloads. We then show how the robot can use rigid body dynamic models and sensing to verify a grasp, to estimate the the inertial parameters of the grasped object, and to adapt the controller and improve performance during flight. We present experimental results with different grippers and different payloads and show the robot's ability to estimate the mass, the location of the center of mass and the moments of inertia to improve tracking performance.

IROS Conference 2011 Conference Paper

Energy-aware coverage control with docking for robot teams

Jason C. Derenick
Nathan Michael
Vijay Kumar 0001

In this paper, we formulate a distributed, energy-aware control policy aimed at enabling persistent surveillance of a specified region of interest by teams of networked robots. Central to our formulation is the fundamental idea that as an agent participating in coverage approaches a low energy reserve the team should cooperatively adjust the coverage formation to allow the agent to return to a designated base station, where it can recharge before rejoining the effort. Towards this end, we build upon recent efforts in employing Centroidal Voronoi Tessellation (CVT)-based coverage control laws by defining a policy that exploits a power-dependent weighting scheme that embeds an agent's trade-off to achieve its coverage mission and to maintain a desired energy reserve to guarantee its own safety. Stability of the proposed approach is considered, and we show that coupling our continuous controller with a straightforward switching mechanism guarantees every agent will return to its base station safely. Simulation results are presented to verify and demonstrate the utility of the proposed control scheme.

IROS Conference 2011 Conference Paper

Localization using ambiguous bearings from radio signal strength

Jason C. Derenick
Jonathan Fink
Vijay Kumar 0001

This paper presents the locomotion approach of a novel quadruped robot which is able to carry various effectors for achieving manufacturing tasks in large workspaces. Equipped with lockers on some of the passive joints and clamping devices at the end of its limbs, this quadruped uses eight actuators for achieving manufacturing tasks as well as locomotion tasks. In the following sections, we first present the proposed robot and its two working modes. Then, the locking strategy of the robot is formulated as an optimization problem. Also, a practical method for managing the limbs swinging movement is addressed. At last, the presented approach is applied on two concrete examples. Possessing a low degree of kinematic redundancy, the proposed quadruped shows a reasonable locomotion capacity which allows it to achieve locomotion with respect to some extra constrains in its workspaces.

ICRA Conference 2011 Conference Paper

Minimum snap trajectory generation and control for quadrotors

Daniel Mellinger
Vijay Kumar 0001

We address the controller design and the trajectory generation for a quadrotor maneuvering in three dimensions in a tightly constrained setting typical of indoor environments. In such settings, it is necessary to allow for significant excursions of the attitude from the hover state and small angle approximations cannot be justified for the roll and pitch. We develop an algorithm that enables the real-time generation of optimal trajectories through a sequence of 3-D positions and yaw angles, while ensuring safe passage through specified corridors and satisfying constraints on velocities, accelerations and inputs. A nonlinear controller ensures the faithful tracking of these trajectories. Experimental results illustrate the application of the method to fast motion (5–10 body lengths/second) in three-dimensional slalom courses.

ICRA Conference 2011 Conference Paper

Recent advances in quadrotor capabilities

Daniel Mellinger
Nathan Michael
Michael Shomin
Vijay Kumar 0001

Over the past year, our capabilities for the quadrotor platform have expanded significantly. This video represents a summary of our developments.

IROS Conference 2011 Conference Paper

Synthesis of feedback controllers for multiple aerial robots with geometric constraints

Nora Ayanian
Vinutha Kallem
Vijay Kumar 0001

We address the problem of developing feedback controllers for a group of robots with second-order dynamics in an obstacle-filled, D-dimensional environment. Our control algorithm takes into account communication constraints, obstacle avoidance, and inter-robot collision avoidance, by synthesizing a piecewise smooth vector field for safe navigation. First, the feasible free joint configuration space is tessellated into polytopes that account for the desired constraints. We search the graph of these polytopes to find a discrete path to the goal polytope. We then use a novel navigation function-based feedback controller that drives the system from one polytope to the next and eventually to the goal. The controller exploits the fact that two adjoining polytopes in the planned discrete path together form a star-shaped object that is obstacle free; this enables the design of navigation function-based controller for kinematic and dynamic fully actuated robots without spurious minima. We sequentially compose these controllers to drive the state to the goal. For a polygonal space, the algorithm we propose is complete. We present successful simulation results of the algorithm on a group of ground vehicles and quadrotors performing a cooperative navigation task in constrained environments.

ICRA Conference 2011 Conference Paper

Time scales and stability in networked multi-robot systems

Mac Schwager
Nathan Michael
Vijay Kumar 0001
Daniela Rus

This paper examines the dynamic interplay between decentralized controllers and mesh networking protocols for controlling groups of robots. A proportional controller is used to maintain robots in a formation based on estimates of the robots' states observed through the network. The state information is propagated through the network using a flooding algorithm, which introduces topology-dependent time delays. The coupled interaction of information flow over the network with the dynamics of the robots is modeled as a linear dynamical system. With this model it is shown that systems made up of robots with stable first order dynamics are stable for all network update times, positive feedback gains, and connected communication graphs. With higher order robot dynamics it is found that stability is a complex and counter intuitive function of feedback gain and network update time. A performance metric is proposed for analyzing the convergence rate of the multi-robot system. Experiments with flying quadrotor robots verify the predictions of the model and the performance metric.

ICRA Conference 2011 Conference Paper

Visibility-based deployment of robot formations for communication maintenance

Ethan Stump
Nathan Michael
Vijay Kumar 0001
Volkan Isler

We consider the problem of deploying robots in formations that ensure network connectivity between a fixed base station and a set of independent agents wandering in the environment. We adopt a communications model that requires line-of-sight and then solve for robot placements by finding mutually-visible configurations in a polygonal decomposition of the environment map. Both the static deployment case and the case of finding deployments that minimize total robot movement are considered. We provide algorithms for the moving agent case, consider their performance on various discretizations for a range of problem sizes, and discuss our experimental implementation of the presented ideas.

ICRA Conference 2011 Conference Paper

Wireless manipulation of single cells using magnetic microtransporters

Mahmut Selman Sakar
Edward B. Steager
Anthony Cowley
Vijay Kumar 0001
George J. Pappas

For such biomedical applications as single cell manipulation and targeted delivery of chemicals, it is important to fabricate microstructures that can be powered and controlled without a tether in fluidic environments. In this work, we describe the construction and operation of micronsized, biocompatible ferromagnetic microtransporters driven by external magnetic fields capable of exerting forces at the pico Newton scale. We develop microtransporters using a simple, single step micro fabrication technique that allows us to produce large numbers in the same step. We also fabricate microgels to deliver drugs. We demonstrate that the microtransporters can be navigated to separate individual targeted cells with micron-scale precision, and deliver microgels without disturbing the cells in the neighborhood and the local microenvironment.

ICRA Conference 2010 Conference Paper

Abstractions and controllers for groups of robots in environments with obstacles

Nora Ayanian
Vijay Kumar 0001

We address the problem of controlling a formation of robots in a cluttered environment. Instead of explicitly controlling the relative positions between the robots and the environment, we construct a lower-dimensional abstraction of the group that establishes a boundary for the group. We then synthesize feedback controllers that allow the abstracted group to navigate a two-dimensional environment to a desired goal position, while automatically adapting the shape of the boundary as well as the position and orientation of the group to avoid collisions between the virtual boundary and the environment. In contrast to previous approaches, we address the planning and control problems concurrently and are naturally able to establish bounds on the positions of the robots through the abstraction. The complexity of the method is independent of the number of robots which promises scalability to large teams.

ICRA Conference 2010 Conference Paper

Biosensing and actuation for microbiorobots

Mahmut Selman Sakar
Edward B. Steager
A. Agung Julius
Min Jun Kim 0001
Vijay Kumar 0001
George J. Pappas

In this paper, we describe how signaling networks and actuation in bacterial cells and biomolecular networks of bacteria can be used to develop an integrated micro-bio-robotic system. SU8 microstructures blotted with swarmer cells of Serratia Marcescens in a monolayer are propelled by the bacteria in the absence of any environmental stimulus. We call such microstructures with bacteria Micro Bio Robots (MBRs) and the uncontrolled motion in the absence of stimuli self actuation. Our paper has two primary contributions. First, we demonstrate the control of MBRs using self-actuation, DC electric fields and ultra-violet radiation, and develop experimentally validated mathematical model for the MBRs. This model allows us to use self-actuation and electrokinetic actuation to steer the MBR to any position and orientation in a planar micro channel. Second, we describe the development of biosensors for the MBRs. This is done by attaching genetically engineered Escherichia coli cells that are capable of sensing nonmetabolizable lactose analog methyl-β-D-thiogalactoside (TMG). We describe the fabrication process for MBRs and show experimental results demonstrating sensing, actuation and control.

ICRA Conference 2010 Conference Paper

Circulation of curves using vector fields: Actual robot experiments in 2D and 3D workspaces

Vinicius Mariano Gonçalves
Luciano C. A. Pimenta
Carlos Andrey Maia
Guilherme A. S. Pereira
Bruno C. O. Dutra
Nathan Michael
Jonathan Fink
Vijay Kumar 0001

Different robotic tasks can be solved by controlling a robot to circulate along curves. These include, for example, border inspection and surveillance, multirobot manipulation, and pattern generation. In a previous, work we have proposed a vector field approach for robot convergence and circulation along time-varying curves embedded in N-dimensional spaces. In the present work we instantiate this approach for three-dimensional spaces and, for the first time, show the efficacy of this method to control actual robots. Besides new theoretical analysis when constant speed control is applied, we present experimental results with aerial (quadrotors) and ground (differential-driven) robot.

ICRA Conference 2010 Conference Paper

Control and planning for vehicles with uncertainty in dynamics

Daniel Mellinger
Vijay Kumar 0001

This paper describes a motion planning algorithm that accounts for uncertainty in the dynamics of vehicles. This noise is a function of the type of controller employed on the vehicle and the characteristics of the terrain and can cause the robot to deviate from a planned trajectory and collide with obstacles. Our motion planning algorithm finds trajectories that balance the trade-off between conventional performance measures such as time and energy versus safety. The key is a characterization of the vehicle's ability to follow planned paths, which allows the algorithm to explicitly calculate probabilities of successful traversal for different trajectory segments. We illustrate the method with a six-legged Rhex-like robot by experimentally characterizing different gaits (controllers) on different terrains and demonstrating the hexapod navigating a multi-terrain environment.

ICRA Conference 2010 Conference Paper

Multi-agent path planning with multiple tasks and distance constraints

Subhrajit Bhattacharya
Maxim Likhachev
Vijay Kumar 0001

The DPC algorithm developed in our previous work is an efficient way of computing optimal trajectories for multiple robots in a distributed fashion with time-parameterized constraints on the distances between pairs of robots. In the present work we extend DPC to the problem of multiple task execution. While this extended problem inherits all the objectives, complexities and constraints of the basic DPC algorithm, each robot is also given an unordered set of tasks that it has to execute before it reaches its goal. There is no specific order imposed on the tasks assigned to a particular robot. The algorithm decides the order of execution of the tasks such that an optimal solution is attained while the time-parametrized distance constraints are satisfied along with successful execution of the tasks. We solve this problem by designing a “State-task Graph” that represents a product of the state-space graph and the task graph. We then develop an efficient heuristic function for performing searches in this graph.

ICRA Conference 2010 Conference Paper

Online methods for radio signal mapping with mobile robots

Jonathan Fink
Vijay Kumar 0001

In this paper we explore methods for the online mapping of received radio signal strength with mobile robots and localizing the source of the radio signal. By utilizing Gaussian processes, we are able to build an online model of the signal-strength map that can, in turn, be used to provide the current maximum likelihood estimate of the source location. Furthermore, using the estimate of the source location, the Gaussian process model allows for prediction of received signal strength with confidence bounds in regions of the environment that have not been explored. Finally, we develop a control law for collecting samples of the signal strength with mobile robots that allows for online estimation of the radio signal source.

SoCS Conference 2010 Conference Paper

Search-Based Path Planning with Homotopy Class Constraints

Subhrajit Bhattacharya
Vijay Kumar 0001
Maxim Likhachev

Goal-directed path planning is one of the basic and widely studied problems in the field of mobile robotics. Homotopy classes of trajectories, arising due to the presence of obstacles, are defined as sets of trajectories that can be transformed into each other by gradual bending and stretching without colliding with obstacles. The problem of finding least-cost paths restricted to a specific homotopy class or finding least-cost paths that do not belong to certain homotopy classes arises frequently in such applications as predicting paths for dynamic entities and computing heuristics for path planning with dynamic constraints. In the present work, we develop a compact way of representing homotopy classes and propose an efficient method of graph search-based optimal path planning with constraints on homotopy classes. The method is based on representing the environment of the robot as a complex plane and making use of the Cauchy Integral Theorem. We prove optimality of the method and show its efficiency experimentally.

ICRA Conference 2010 Conference Paper

Towards simplicial coverage repair for mobile robot teams

Jason C. Derenick
Vijay Kumar 0001
Ali Jadbabaie

In this note, we present initial results towards developing a distributed algorithm for repairing topological holes in the sensor cover of a mobile robot team. Central to our approach is the melding of recent advances in the application of computational homology (a sub-discipline of algebraic topology) to static sensor networks with relative metric information (i. e. relative pose). More precisely, we consider a greedy, hybrid (discrete-continuous) algorithm whereby a desired Cěch complex, the simplicial complex that captures the underlying topology of the sensing cover, is iteratively generated using local rules (between multi-hop neighbors) and agents are driven towards achieving this topology via a gradient-ascent simplicial control law. Convergence of the proposed algorithm is established as a function of the convergence of the underlying simplicial control law, and the relationship of the latter to the spectrum of the combinatorial Laplacian is considered. Simulation results for teams operating in ℝ 2 are presented.

ICRA Conference 2009 Conference Paper

Control of locomotion with shape-changing wheels

Daniel Mellinger
Vijay Kumar 0001
Mark Yim

We present a novel approach to controlling the locomotion of a wheel by changing its shape, leading to applications to the synthesis and closed-loop control of gaits for modular robots. A dynamic model of a planar, continuous deformable ellipse in contact with a ground surface is derived. We present two alternative approaches to controlling this system and a method for mapping the gaits to a discrete rolling polygon. Mathematical models and dynamic simulation of the continuous approximation and the discrete n-body system, and experimental results obtained from a physical modular robot system illustrate the accuracy of the dynamic models and the validity of the approach.

IROS Conference 2009 Conference Paper

Experimental characterization of radio signal propagation in indoor environments with application to estimation and control

Jonathan Fink
Nathan Michael
Aleksandr Kushleyev
Vijay Kumar 0001

We study radio signal propagation in indoor environments using low-power devices leveraging the Zigbee and Bluetooth specifications. We present results from experiments where two robots equipped with radio signal devices and enabled to control and localize autonomously in an indoor hallway and laboratory environment densely sample RSSI at various times over several days. We show that simulated RSSI measurements using existing radio signal models and experimentally gathered RSSI measurements match closely, suggesting that for robotics applications requiring predicted RSSI, low-power radio signal devices are a well-posed sensing modality.

ICRA Conference 2009 Conference Paper

Harnessing bacterial power in microscale actuation

A. Agung Julius
Mahmut Selman Sakar
Edward B. Steager
U. Kei Cheang
Min Jun Kim 0001
Vijay Kumar 0001
George J. Pappas

This paper presents a systematic analysis of the motion of microscale structures actuated by flagellated bacteria. We perform the study both experimentally and theoretically. We use a blotting procedure to attach flagellated bacteria to a buoyancy-neutral plate called a microbarge. The motion of the plate depends on the distribution of the cells on the plate and the stimuli from the environment. We construct a stochastic mathematical model for the system, based on the assumption that the behavior of each bacterium is random and independent of that of its neighbors. The main finding of the paper is that the motion of the barge plus bacteria system is a function of a very small set of parameters. This reduced-dimensional model can be easily estimated using experimental data. We show that the simulation results obtained from the model show an excellent match with the experimentally-observed motion of the barge.

ICRA Conference 2009 Conference Paper

Stochastic strategies for a swarm robotic assembly system

Loic Matthey
Spring Berman
Vijay Kumar 0001

We present a decentralized, scalable approach to assembling a group of heterogeneous parts into different products using a swarm of robots. While the assembly plans are predetermined, the exact sequence of assembly of parts and the allocation of subassembly tasks to robots are determined by the interactions between robots in a decentralized fashion in real time. Our approach is based on developing a continuous abstraction of the system derived from models of chemical reactions and formulating the strategy as a problem of selecting rates of assembly and disassembly. These rates are mapped onto probabilities that determine stochastic control policies for individual robots, which then produce the desired aggregate behavior. This top-down approach to determining robot controllers also allows us to optimize the rates at the abstract level to achieve fast convergence to the specified target numbers of products. Because the method incorporates programs for assembly and disassembly, changes in demand can lead to reconfiguration in a seamless fashion. We illustrate the methodology using a physics-based simulator with examples involving 15 robots and two types of final products.

ICRA Conference 2009 Conference Paper

Two-dimensional, vision-based muN force sensor for microrobotics

David J. Cappelleri
Gianluca Piazza
Vijay Kumar 0001

We present a two-dimensional, vision-based force sensor, capable of sensing muN level forces. There are currently no reliable, off-the-shelf, commercially-available force sensors to measure forces at this scale, that can be easily integrated into standard microrobotic test-beds. Our design consists of a planar, elastic mechanism with known force-deflection characteristics. A CCD camera is used to track the deformation of the mechanism as it is used to manipulate objects in a micro/meso-scale robotic manipulation test-bed. By observing the displacements of select points in the mechanism, the manipulation forces can be estimated. The modeling, design, microfabrication, calibration and experimental validation of the force sensor are presented with a brief discussion of an application to a robotic manipulation tasks such as microassembly.

ICRA Conference 2008 Conference Paper

An almost communication-less approach to task allocation for multiple Unmanned Aerial Vehicles

Peng Cheng 0009
Vijay Kumar 0001

In this paper, we present a scalable, decentralized task allocation algorithm for a group of unknown number of unmanned aerial vehicles (UAVs), which are equipped with GPS receivers, synchronized clocks and radars with a finite, but known operating distance to identify neighbors. The algorithm assigns subgroups of UAVs, whose initial positions are randomly scattered in a bounded space, to a finite set of independent tasks. The key features of the proposed algorithm are: (1) the algorithm does not require any communication between the UAVs; (2) the task allocation is achieved in finite time. The analysis and results in the simplified 2D simulation environment respectively prove and verify the correctness of the proposed algorithm.

ICRA Conference 2008 Conference Paper

Connectivity management in mobile robot teams

Ethan Stump
Ali Jadbabaie
Vijay Kumar 0001

We develop a framework for controlling a team of robots to maintain and improve a communication bridge between a stationary robot and an independently exploring robot in a walled environment. We make use of two metrics for characterizing the communication: the Fiedler value of the weighted Laplacian describing the communication interactions of all the robots in the system, and the k-connectivity matrix that expresses which robots can interact through k or less intermediary robots. At each step, we move in such a way as to improve the Fiedler value as much as possible while keeping the number of intermediary robots between the two robots of interest below a desired value. We demonstrate the use of this framework in a scenario where the hop-count constraint cannot be satisfied, but show that communication quality is maintained anyways.

ICRA Conference 2008 Conference Paper

Control of swarms based on Hydrodynamic models

Luciano C. A. Pimenta
Nathan Michael
Renato Cardoso Mesquita
Guilherme A. S. Pereira
Vijay Kumar 0001

We address the problem of pattern generation in obstacle-filled environments by a swarm of mobile robots. Decentralized controllers are devised by using the Smoothed Particle Hydrodynamics (SPH) method. The swarm is modelled as an incompressible fluid subjected to external forces. Actual robot issues such as finite size and nonholonomic constraints are also addressed. Collision avoidance guarantees are discussed. Finally, in the absence of obstacles, we prove for the first time stability and convergence of controllers based on the SPH.

ICRA Conference 2008 Conference Paper

Decentralized feedback controllers for multi-agent teams in environments with obstacles

Nora Ayanian
Vijay Kumar 0001

We propose a method for synthesizing decentralized feedback controllers for a team of multiple heterogeneous agents navigating a known environment with obstacles. The controllers are designed to drive agents with limited team state information to goal sets while avoiding collisions and maintaining specified proximity constraints. The method, its successful application to nonholonomic agents in dynamic simulation, and its limitations are presented in this paper.

ICRA Conference 2008 Conference Paper

Distributed multi-robot task assignment and formation control

Nathan Michael
Michael M. Zavlanos
Vijay Kumar 0001
George J. Pappas

Distributed task assignment for multiple agents raises fundamental and novel problems in control theory and robotics. A new challenge is the development of distributed algorithms that dynamically assign tasks to multiple agents, not relying on a priori assignment information. We address this challenge using market-based coordination protocols where the agents are able to bid for task assignment with the assumption that every agent has knowledge of the maximum number of agents that any given task can accommodate. We show that our approach always achieves the desired assignment of agents to tasks after exploring at most a polynomial number of assignments, dramatically reducing the combinatorial nature of discrete assignment problems. We verify our algorithm through both simulation and experimentation on a team of non-holonomic robots performing distributed formation stabilization and group splitting and merging.

ICRA Conference 2008 Conference Paper

Meso-scale manipulation: System, modeling, planning and control

David J. Cappelleri
Peng Cheng 0009
Jonathan Fink
Bogdan Gavrea
Vijay Kumar 0001

Manipulation and assembly tasks are typically characterized by many nominally rigid bodies coming into frictional contacts, possibly involving impacts. Manipulation tasks are difficult to model because uncertainties associated with friction and assembly tasks are particularly hard to analyze because of the interplay between process tolerance and geometric uncertainties due to manufacturing errors. Manipulation at the meso (hundred microns to millimeters) and micro (several microns to tens of microns) scale is even harder for several reasons. It is difficult to measure forces at the micro-netwon level reliably using off-the-shelf force sensors and good force-feedback control schemes have not proved successful. It is hard to manufacture general-purpose end effectors at this scale and it is even more difficult to grasp and manipulate parts at the micro and meso level than it is at the macro level. Finally, the lack of good models of the mechanics of contact interactions at this scale means that model-based approaches to planning and control are difficult.

ICRA Conference 2008 Conference Paper

Multi-robot manipulation via caging in environments with obstacles

Jonathan Fink
M. Ani Hsieh
Vijay Kumar 0001

We present a decentralized approach to multirobot manipulation where the team of robots surround and trap an object and transport it, by dragging or pushing, to the goal configuration in an environment with obstacles. The proposed feedback controllers are obtained by sequentially composing vector fields or behaviors and are decentralized in the sense that robots do not exchange each other’s state information. Rather, cooperative manipulation is achieved by relying solely on each robot’s local information and a global knowledge of the task. We present computer simulations and experimental results obtained using our multi-robot testbed.

IROS Conference 2008 Conference Paper

Time-optimal UAV trajectory planning for 3D urban structure coverage

Peng Cheng 0009
James Keller 0002
Vijay Kumar 0001

In this paper, we study the time-optimal trajectory planning of a sensor attached to an unmanned aerial vehicle (UAV) to provide complete 3-dimensional coverage with applications to urban environments with 2. 5-dimensional features. The basic approach is to approximate the features of interest with a set of non planar coverage surfaces and to design a motion plan that guarantees the coverage surface is swept completely with a conical-field-of-view sensor. We establish a lower bound on time for a UAV to achieve complete coverage and derive the analytical coverage plan whose duration is a constant times this lower bound. Our hardware-in-the-loop simulation results verify the effectiveness of the proposed algorithm.

ICRA Conference 2007 Conference Paper

Bio-Inspired Group Behaviors for the Deployment of a Swarm of Robots to Multiple Destinations

Spring Berman
Ádám M. Halász
Vijay Kumar 0001
Stephen Pratt

We present a methodology for characterizing and synthesizing swarm behaviors using both a macroscopic model that represents a swarm as a continuum and a microscopic model that represents individual robots. We develop a systematic approach for synthesizing behaviors at the macroscopic level that can be realized on individual robots at the microscopic level. Our methodology is inspired by a dynamical model of ant house hunting [1], a decentralized process in which a colony attempts to emigrate to the best site among several alternatives. The model is hybrid because the colony switches between different sets of behaviors, or modes, during this process. At the macroscopic level, we are able to synthesize controllers that result in the deployment of a robotic swarm in a predefined ratio between distinct sites. We then derive hybrid controllers for individual robots using only local interactions and no communication that respect the specifications of the global continuous behavior. Our simulations demonstrate that our synthesis procedure yields a correct microscopic model from the macroscopic description with guarantees on performance at both levels

IROS Conference 2007 Conference Paper

Controlling a team of ground robots via an aerial robot

Nathan Michael
Jonathan Fink
Vijay Kumar 0001

We consider the task of controlling a large team of nonholonomic ground robots with an unmanned aerial vehicle in a decentralized manner that is invariant to the number of ground robots. The central idea is the development of an abstraction for the team of ground robots that allows the aerial platform to control the team without any knowledge of the specificity of individual vehicles. This happens in much the same way as a human operator can control a single robot vehicle by simply commanding the forward and turning velocities without a detailed knowledge of the specifics of the robot. The abstraction includes a gross model of the shape of the formation of the team and information about the position and orientation of the team in the plane. We derive controllers that allow the team of robots to move in formation while avoiding collisions and respecting the abstraction commanded by the aerial platform. We provide simulation and experimental results using a team of indoor mobile robots and a three-dimensional, cable-controlled, parallel robot which serves as our indoor unmanned aerial platform.

ICRA Conference 2007 Conference Paper

daVinci Code: A Multi-Model Simulation and Analysis Tool for Multi-Body Systems

Stephen Berard
Jeffrey C. Trinkle
Binh Nguyen 0002
Ben Roghani
Jonathan Fink
Vijay Kumar 0001

This paper discusses the design and current capabilities of a new software tool, dVC, capable of simulating planar systems of bodies experiencing unilateral contacts with friction. Since different problems require different levels of accuracy, dVC provides user-selectable body types (rigid or locally-compliant), motion models (first-order, quasi-static, dynamic), and several state-of-the-art time-stepping methods. One can also choose to include friction between each body and the plane of motion. To support optimal and robust part design, dVC also allows on-the-fly changes to parameters of the geometric and physical models. The results obtained for three representative planar problems are presented: the design of a passive part-orienting device, the planning of a mesoscale assembly operation, and the design of a grasp strategy.

ICRA Conference 2007 Conference Paper

Decidability of Motion Planning with Differential Constraints

Peng Cheng 0009
George J. Pappas
Vijay Kumar 0001

Classical path planning does not address many of the challenges of robotic systems subject to differential constraints. While there have been many recent efforts to develop motion planning algorithms for systems with differential constraints (MPD), very little has been said about the existence of exact algorithms. In other words, the decidability of MPD problems is still an open question. In this paper, we propose a partial answer to this question limiting ourselves to special cases where the trajectory functions of the systems under the finite-dimensional piecewise-continuous controls have a closed-form polynomial formulation. We define an abstract formulation for the MPD problem based on the concept of a control space. We provide an incremental decision algorithm to answer the decidability question and present sufficient conditions for problems to which this algorithm can be applied. Decidability results for several non trivial MPD problems are presented. For example, we show that the question of existence of a trajectory for a Dubin's car with a polygonal rigid body between two specified positions and orientations in a polygonal environment with a fixed and finite number of discontinuities in curvature is decidable.

IROS Conference 2007 Conference Paper

Dynamic redistribution of a swarm of robots among multiple sites

Ádám M. Halász
M. Ani Hsieh
Spring Berman
Vijay Kumar 0001

We present an approach for the dynamic assignment and reassignment of a large team of homogeneous robotic agents to multiple locations with applications to search and rescue, reconnaissance and exploration missions. Our work is inspired by experimental studies of ant house hunting and empirical models that predict the behavior of the colony that is faced with a choice between multiple candidate nests. We design stochastic control policies that enable the team of agents to distribute themselves between multiple candidate sites in a specified ratio. Additionally, we present an extension to our model to enable fast convergence via switching behaviors based on quorum sensing. The stability and convergence properties of these control policies are analyzed and simulation results are presented.

ICRA Conference 2007 Conference Paper

Mixed Initiative Control of Autonomous Vehicles

Savvas G. Loizou
Vijay Kumar 0001

While there is extensive work on motion planning and control for navigation tasks with guarantees, there is no systematic way for human operators to modify the resulting plans without losing the guarantees. In this paper we propose a systematic way of composing behaviors resulting from human inputs with behaviors derived from navigation functions. The proposed controller is based on a new class of navigation function based controllers that possess weak input-to-state stability properties. The resulting system has analytically guaranteed safety and convergence properties. The feasibility of the proposed methodology is demonstrated through simulation examples and hardware experiments

ICRA Conference 2007 Conference Paper

Stabilization of Multiple Robots on Stable Orbits via Local Sensing

M. Ani Hsieh
Savvas G. Loizou
Vijay Kumar 0001

We develop decentralized controllers for a team of disk-shaped robots to converge to and circulate along the boundary of a desired two-dimensional geometric pattern specified by a smooth function with collision avoidance. The proposed feedback controllers rely solely on each robot's range and bearing sensors which allow them to obtain information about positions of neighbors within a given range. This is relevant for applications such as perimeter surveillance or containing hazardous regions where limited bandwidth must be preserved for situational awareness. The computational complexity of the decentralized controller for each agent is linear in the number of neighboring agents, making it scalable to robot swarms. We establish stability and convergence properties of the controllers and verify the feasibility of the method through computer simulations.

ICRA Conference 2006 Conference Paper

Controlling three Dimensional Swarms of Robots

Nathan Michael
Calin Belta
Vijay Kumar 0001

We address the problem of modeling and controlling a swarm of fully actuated point-like robots in three dimensions by generalizing the planar framework from (C. Belta and V. Kumar, 2004). We define a nine-dimensional abstraction of the swarm that has a product structure of the six-dimensional Euclidean group and a three-dimensional shape, and is independent of the number of robots. The group captures the pose of an ellipsoid spanning the swarm with semiaxes given by the shape variables. The overall abstract description is invariant to robot permutations. In addition, the shape is also invariant to left actions of the group. This description allows one to define and control the behavior of the swarm at a high level, with automatic generation of individual robot control laws. We present simulation results for controlling swarms of rotorcrafts

ICRA Conference 2006 Conference Paper

Designing Open-loop Plans for Planar Micro-manipulation

David J. Cappelleri
Jonathan Fink
Barry Munkundakrisnam
Vijay Kumar 0001
Jeffrey C. Trinkle

This paper describes a test-bed for planar micro manipulation tasks and a framework for planning based on quasi-static models of mechanical systems with frictional contacts. We show how planar peg-in-the-hole assembly tasks can be designed using randomized motion planning techniques with Mason's models for quasi-static manipulation. Finally, we present simulation and experimental results in support of our methodology

ICRA Conference 2006 Conference Paper

Motion Planning for the Roller Racer with a Sticking/Slipping Switching Model

Peng Cheng 0009
Emilio Frazzoli
Vijay Kumar 0001

The roller racer, an undulatory locomotion system, is a toy which can be propelled forward by sitting on it and only oscillating the steering handle. A nonholonomic dynamic model and controllability analysis of the roller racer was first published by Krishnaprasad and Tsakiris in 1998. The model is derived from the usual assumption that all the wheels obey sticking (non-slipping) constraints, i. e. , rolling without slipping. Controllability analysis shows that under these assumptions, the roller racer cannot be stopped once started. Yet physical prototypes do not exhibit this characteristic. In this paper, a high-fidelity model of the roller racer is presented by considering the finite static friction between the wheels and the ground, i. e. , slipping will occur when constraint force exceeds the maximal allowable frictional force. It is proved that the system could be stopped from any state with only the steering angle control. Furthermore, based on group symmetry and motion primitives, a planner is designed to achieve motions between any two given positions and orientations with zero velocities. Experiments also show that front wheel slipping stops the system faster than joint frictions

ICRA Conference 2006 Conference Paper

Pattern Generation with Multiple Robots

M. Ani Hsieh
Vijay Kumar 0001

We develop decentralized controllers for a swarm of robots to generate a desired two-dimensional geometric pattern specified by a smooth function while maintaining specified relative distance constraints. The controllers are decentralized in the sense that the robots do not exchange or sense each other's state information. However, we assume that the robots have range sensors allowing them to obtain information about distances to neighbors within a known range. We establish stability and convergence properties of the controllers

ICRA Conference 2006 Conference Paper

Towards the Deployment of a Mobile Robot Network with End-to-end Performance Guarantees

M. Ani Hsieh
Anthony Cowley
Vijay Kumar 0001
Camillo J. Taylor

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we present an experimental study of strategies for maintaining end-to-end communication links for tasks such as surveillance and search and rescue where team connectivity is essential for providing situational awareness to a base station. We consider the differences between monitoring point-to-point signal strength versus data throughput and present experimental results with our multi-robot testbed in outdoor environments

ICRA Conference 2005 Conference Paper

Controlling Swarms of Robots Using Interpolated Implicit Functions

Luiz Chaimowicz
Nathan Michael
Vijay Kumar 0001

We address the synthesis of controllers for large groups of robots and sensors, tackling the specific problem of controlling a swarm of robots to generate patterns specified by implicit functions of the form s(x, y) = 0. We derive decentralized controllers that allow the robots to converge to a given curve S and spread along this curve. We consider implicit functions that are weighted sums of radial basis functions created by interpolating from a set of constraint points, which give us a high degree of control over the desired 2D curves. We describe the generation of simple plans for swarms of robots using these functions and illustrate our approach through simulations and real experiments.

ICRA Conference 2005 Conference Paper

Information Driven Coordinated Air-Ground Proactive Sensing

Ben Grocholsky
Rahul Swaminathan
James Keller 0002
Vijay Kumar 0001
George J. Pappas

This paper concerns the problem of actively searching for and localizing ground features by a coordinated team of air and ground robotic sensor platforms. The approach taken builds on well known Decentralized Data Fusion (DDF) methodology. In particular, it brings together established representations developed for identification and linearized estimation problems to jointly address feature detection and localization. This provides transparent and scalable integration of sensor information from air and ground platforms. As in previous studies, an Information-theoretic utility measure and local control strategy drive the robots to uncertainty reducing team configurations. Complementary characteristics in terms of coverage and accuracy are revealed through analysis of the observation uncertainty for air and ground on-board cameras. Implementation results for a detection and localization example indicate the ability of this approach to scalably and efficiently realize such collaborative potential.

ICRA Conference 2005 Conference Paper

Planning and Control of Mobile Robots in Image Space from Overhead Cameras

Rahul Rao
Vijay Kumar 0001
Camillo J. Taylor

In this work, we present a framework for the development of a planar mobile robot controller based on image plane feedback. We show that the design of such a motion controller can be accomplished in the image plane by making use of a subset of the parameters that relate the image plane to the ground plane, while still leveraging the simplifications offered by modeling the system as a differentially flat system. Our method relies on a waypoint-based trajectory generator, with all the waypoints specified in the image, as seen by an overhead observer. We present some results from simulation as well as from experiments that validate the ideas presented in this work and discuss some ideas for future work.

ICRA Conference 2005 Conference Paper

RoboTrikke: A Novel Undulatory Locomotion System

Sachin Chitta
Peng Cheng 0009
Emilio Frazzoli
Vijay Kumar 0001

The TRIKKE is a three-wheeled, human-powered scooter that can be propelled by a combination of cyclic motion of its handlebar and swaying motion of the rider. This paper addresses the modeling, dynamics and control of the TRIKKE and the development of a robotic platform called the ROBOTRIKKE that is derived from similar principles. The TRIKKE can be modeled as a modified roller-racer with an unstable steering arrangement. The model of the TRIKKE reduces to the roller-racer [8] in the absence of this steering arrangement. We prove that under certain conditions on the geometric parameters of the system, the TRIKKE and roller-racer systems cannot be stopped after motion starting from rest using the steering control as the sole input. As a consequence, the ideal model is severely limited from the point of view of controllability. We demonstrate the validity of our model through comparison with experimental measurements on a small-scale robotic prototype of the TRIKKE. We present closed-loop control results for tracking (on average) a straight line trajectory using visual feedback from an overhead camera.

ICRA Conference 2005 Conference Paper

Usability Study of a Control Framework for an Intelligent Wheelchair

Sarangi P. Parikh
Valdir Grassi Jr.
Vijay Kumar 0001
Jun Okamoto Jr.

We describe the development and assessment of a computer controlled wheelchair called the SMARTCHAIR. A shared control framework with different levels of autonomy allows the human operator to stay in complete control of the chair at each level while ensuring her safety. The framework incorporates deliberative motion plans or controllers, reactive behaviors, and human user inputs. At every instant in time, control inputs from these three different sources are blended continuously to provide a safe trajectory to the destination, while allowing the human to maintain control and safely override the autonomous behavior. In this paper, we present usability experiments with 50 participants and demonstrate quantitatively the benefits of human-robot augmentation.

ICRA Conference 2004 Conference Paper

Constructing Radio Signal Strength Maps with Multiple Robots

M. Ani Hsieh
Vijay Kumar 0001
Camillo J. Taylor

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we investigate the construction of a map of radio signal strength that can be used to plan multirobot tasks and also serve as useful perceptual information. We show how nominal models of an urban environment, such as those obtained by aerial surveillance, can be used to generate strategies for exploration and present preliminary experimental results with our multi-robot testbed.

ICRA Conference 2004 Conference Paper

Design and Gait Control of a Rollerblading Robot

Sachin Chitta
Frederik W. Heger
Vijay Kumar 0001

We present the design and gait generation for an experimental ROLLERBLADER. The ROLLERBLADER is a robot with a central platform mounted on omnidirectional casters and two 3 degree-of-freedom legs. A passive rollerblading wheel is attached to the end of each leg. The wheels give rise to nonholonomic constraints acting on the robot. The legs can be picked up and placed back on the ground allowing a combination of skating and walking gaits. We present two types of gaits for the robot. In the first gait, we allow the legs to be picked up and placed back on the ground while in the second, the wheels are constrained to stay on the ground at all tunes. Experimental gait results for a prototype robot are also presented.

ICRA Conference 2004 Conference Paper

Design of Part Feeding and Assembly Processes with Dynamics

Peng Song 0005
Jeffrey C. Trinkle
Vijay Kumar 0001
Jong-Shi Pang

We introduce computational support tools for the analysis and design of systems with multiple frictional contacts, with a focus on applications to part feeding and assembly processes. The tools rely on dynamic models of the processes. We describe two approaches to modeling, the Stewart-Trinkle model (1996) and the Song-Pang-Kumar model (2003), that allow the designer to experiment with different geometric, material and dynamic properties and optimize the design for performance. In order to accommodate contact transitions, we introduce a smooth cone model for friction. We illustrate the models and the design process by describing the design optimization of a part feeder.

ICRA Conference 2004 Conference Paper

Experiments in Multirobot Air-Ground Coordination

Luiz Chaimowicz
Ben Grocholsky
James Keller 0002
Vijay Kumar 0001
Camillo J. Taylor

This paper addresses the problem of coordinating aerial and ground vehicles in tasks that involve exploration, identification of targets and maintaining a connected communication network. We focus on the problem of localizing vehicles in urban environments where GPS signals are often unreliable or unavailable. We first describe our multi-robot testbed and the control software used to coordinate ground and aerial vehicles. We present the results of experiments in air-ground localization analyzing three complementary approaches to determining the positions of vehicles on the ground. We show that the coordination of aerial vehicles with ground vehicles is necessary to get accurate estimates of the state of the system.

ICRA Conference 2004 Conference Paper

Formations for Localization of Robot Networks

Fan Zhang
Ben Grocholsky
Vijay Kumar 0001

In this paper, we consider the problem of cooperatively localizing a formation of networked mobile robots/vehicles in SE(2), and adapting the formation to reduce localization errors. First, we propose necessary and sufficient conditions to establish when a team of robots with heterogeneous sensors can be completely localized. We present experimental measurements of range and bearing with omni-directional cameras to motivate a simple model for noisy sensory information. We propose a measure of quality of team localization, and show how this measure directly depends on a sensing graph. Finally, we show how the formation and the sensing graph can be adapted to improve the measure of performance for team localization and for localization of targets through experiments and simulations.

ICRA Conference 2004 Conference Paper

Incorporating User Inputs in Motion Planning for a Smart Wheelchair

Sarangi P. Parikh
Valdir Grassi Jr.
Vijay Kumar 0001
Jun Okamoto Jr.

We describe the development and assessment of a computer controlled wheelchair equipped with a suite of sensors and a novel interface, called the SMARTCHAIR. The main focus of this paper is a shared control framework which allows the human operator to interact with the chair while it is performing an autonomous task. At the highest level, the autonomous system is able to plan paths using high level deliberative navigation behaviors depending on destinations or waypoints commanded by the user. The user is able to locally modify or override previously commanded autonomous behaviors or plans. This is possible because of our hierarchical control strategy that combines three independent sources of control inputs: deliberative plans obtained from maps and user commands, reactive behaviors generated by stimuli from the environment, and user-initiated commands that might arise during the execution of a plan or behavior. The framework we describe ensures the user's safety while allowing the user to be in complete control of a potentially autonomous system.

ICRA Conference 2003 Conference Paper

Abstraction and control for groups of fully-actuated planar robots

Calin Belta
Vijay Kumar 0001

This paper shows how a large number of robots can be coordinated by designing control laws on a small dimensional manifold, independent on the number and ordering of the robots. The small dimensional description of the team has a product structure of a Lie group, which captures the dependence of the ensemble on world frame, and a shape manifold, which is an intrinsic description of the team. We design decoupled controls for group and shape. The individual control laws which are mapped to the desired collective behavior can be realized by feedback depending only on the current state of the robot and the state on the small dimensional manifold, so that the robots have to broadcast their states and only have to listen to some coordinating agent with small bandwidth.

IROS Conference 2003 Conference Paper

Closed loop navigation for mobile agents in dynamic environments

Savvas G. Loizou
Herbert G. Tanner
Vijay Kumar 0001
Kostas J. Kyriakopoulos

We apply a novel motion planning and control methodology, which is based on a non-smooth navigation function, to a point mobile robot moving amongst moving obstacles. The chattering introduced by the discontinuous potential field is suppressed using non-smooth backstepping. The combined controller guarantees global asymptotic convergence and collision avoidance. This controller is particularly suitable for real time implementation on systems with limited computational resources. The effectiveness of the proposed scheme is verified through computer simulations.

IROS Conference 2003 Conference Paper

Design and verification of controllers for airships

Jongwoo Kim
James Keller 0002
Vijay Kumar 0001

Robotic airships have several beneficial properties such as low operation cost, low noise, and low speed flight capability. We present in this paper the design and verification of a feedback control algorithm for waypoint to waypoint navigation of an outdoor blimp. A rapidly exploring random tree (RRT) is used for the validation of the blimp system control law. We describe an implementation of an algorithm that systematically searches the set of all disturbances to validate viability of the control law in the presence of winds. Experimental results with a simulator show that the RRT method can be effective in verifying controller design under unpredictable but bounded disturbances.

IROS Conference 2003 Conference Paper

Dynamics and generation of gaits for a planar rollerblader

Sachin Chitta
Vijay Kumar 0001

We develop the dynamic model for a planar rollerblader. The robot consists of a rigid platform and two planar, two degree-of-freedom legs with in-line skates at the foot. The dynamic model consists of two unicycles coupled through the rigid body dynamics of the planar platform. We derive the Lagrangian reduction for the rollerblading robot. We show the generation of some simple gaits that allow the platform to move forward and rotate by using cyclic motions of the two legs.

IROS Conference 2003 Conference Paper

Formation control with configuration space constraints

Guilherme A. S. Pereira
Aveek K. Das
Vijay Kumar 0001
Mario F. M. Campos

We address the problem of controlling a team of robots subject to constraints on relative positions. We adopt the general framework of leader-follower control in which a network of controllers is used to control the position and orientation of the team and its shape. We propose two improvements to this scheme. First, we introduce cooperative leader-following where the motion of a robot is determined not only by its leader, but also by other robots including their followers. Second, we allow constraints that are induced by limitations on ranges of sensors and wireless network cards. Our approach is based on potential field controllers for each robot and the on-line modification of these controllers to accommodate motion constraints induced by other robots in the group. We present experimental results with a team of three car-like robots equipped with omnidirectional cameras and 802. 11b network cards.

IROS Conference 2003 Conference Paper

Human robot interaction and usability studies for a smart wheelchair

Sarangi P. Parikh
Rahul Rao
Sang-Hack Jung
Vijay Kumar 0001
Jim Ostrowski 0001
Camillo J. Taylor

We build on previous work, on the development of a computer controlled wheelchair equipped with a suite of sensors and a novel interface for human-robot interaction. In this paper, we present experimental results and usability studies for the wheelchair. The architecture for human-robot interaction is hierarchical, with the lowest level corresponding to trajectory control, the intermediate level being behavioral and the highest level involving the composition of behaviors and navigation. Our experimental results illustrate the benefits of a shared-control paradigm where the human operator selects the appropriate behavior(s) or goals while the software is responsible for executing behaviors and generating safe trajectories. Experiments with human users highlight advantages of augmentation in wheelchairs.

ICRA Conference 2003 Conference Paper

Hybrid systems modeling of cooperative robots

Luiz Chaimowicz
Mario F. M. Campos
Vijay Kumar 0001

This paper proposes a methodology that uses hybrid systems to model multiple robots in the execution of cooperative tasks. Basically, each robot is represented by a hybrid automation and the cooperative task execution is modeled by the composition of several automata. We describe in details our approach to perform the composition of these automata and demonstrate the effectiveness of the proposed methodology modeling a cooperative manipulation task.

IROS Conference 2003 Conference Paper

Using policy gradient reinforcement learning on autonomous robot controllers

Gregory Z. Grudic
Vijay Kumar 0001
Lyle H. Ungar

Robot programmers can often quickly program a robot to approximately execute a task under specific environment conditions. However, achieving robust performance under more general conditions is significantly more difficult. We propose a framework that starts with an existing control system and uses reinforcement feedback from the environment to autonomously improve the controller's performance. We use the policy gradient reinforcement learning (PGRL) framework, which estimates a gradient (in controller space) of improved reward, allowing the controller parameters to be incrementally updated to autonomously achieve locally optimal performance. Our approach is experimentally verified on a Cye robot executing a room entry and observation task, showing significant reduction in task execution time and robustness with respect to un-modelled changes in the environment.

IROS Conference 2003 Conference Paper

Visual servoing of a UGV from a UAV using differential flatness

Rahul Rao
Vijay Kumar 0001
Camillo J. Taylor

In this paper the problem of controlling the motion of a nonholonomic vehicle along a desired trajectory using observations from an overhead camera is considered. The control problem is formulated in the image plane. We show that the system in the image plane is differentially flat and use this property to generate effective control strategies using only visual feedback. Simulation results illustrate the methodology and show robustness to errors in the camera calibration parameters.

ICRA Conference 2002 Conference Paper

A Method for Modifying Closed-Loop Motion Plans to Satisfy Unpredictable Dynamic Constraints at Runtime

Joel M. Esposito
Vijay Kumar 0001

The problem of motion planning in environments with both known static obstacles and unpredictable dynamic constraints is considered. A methodology is introduced in which the motion plan for the static environment is modified on-line to accommodate the unpredictable constraints in such a way that the completeness properties of the original motion plan are preserved. At the heart of the approach is the idea that navigation functions are indeed Lyapunov functions; and that the traditional method of forcing the robot to track the negative gradient of field is not the only input which stabilizes the system. This extra freedom in selecting the input is used to accommodate the dynamic constraints. A computational method for selecting the appropriate inputs is given. The method is used to solve two sample problems. The constraints in these cases are used to model collisions with other robots and, in the second example, a team of robots traveling information. Finally, some preliminary work on extending the approach to nonholonomic systems is presented.

ICRA Conference 2002 Conference Paper

A Potential Field Based Approach to Multi-Robot Manipulation

Peng Song 0005
Vijay Kumar 0001

We describe a framework for controlling and coordinating a group of robots for cooperative manipulation tasks. The framework enables a decentralized approach to planning and control. It allows the robots to approach the object, organize themselves into a formation that will trap the object, and then transport the object to the desired destination. Our controllers and planners are derived from simple potential fields and the hierarchical composition of potential fields. We show how these potential field based controllers and planners benefit complex group interactions, specifically for manipulating and transporting objects in the plane. Theoretically, we show how we can derive results on formation stability with potential field based controllers in many cases. Simulation results demonstrate successful application to a wide range of examples without showing sensitivity to parameters. Because the framework is decentralized at both trajectory generation level and the estimation and control agent level, our framework can potentially scale to groups of tens and hundreds of robots.

ICRA Conference 2002 Conference Paper

Dynamic Role Assignment for Cooperative Robots

Luiz Chaimowicz
Mario F. M. Campos
Vijay Kumar 0001

Proposes a methodology for coordinating multi-robot teams in the execution of cooperative tasks. It is based on a dynamic role-assignment mechanism in which the robots assume and exchange roles during cooperation. We model the role assignment under a hybrid systems framework, using a hybrid automaton to represent roles, transitions and controllers. Using a multi-robot simulator, the methodology is demonstrated in a cooperative transportation task, in which a group of robots must find and cooperatively transport several objects scattered in the environment.

ICRA Conference 2002 Conference Paper

Human Robot Interaction: Application to Smart Wheelchairs

R. S. Rao
K. Conn
Sang-Hack Jung
Jay Katupitiya
Terry Kientz
Vijay Kumar 0001
Jim Ostrowski 0001
Sarangi Patel

Addresses the problem of human robot interaction with application to the design of assistive devices. We describe the design and development of a prototype of a smart wheelchair that can be commanded by a rider. Specifically, we focus on (a) the vision-based human interaction interface; (b) the suite of sensors on the chair, and (c) the software architecture and the control algorithms used to control the chair.

ICRA Conference 2002 Conference Paper

The Effect of Feedback and Feedforward on Formation ISS

Herbert G. Tanner
Vijay Kumar 0001
George J. Pappas

A new type of stability of leader follower formations is defined, based on input-to-state stability (ISS) properties of cascade interconnections. Formation ISS links leader input to internal state of the formation and characterizes the way this input affects performance. The effect of feedforward and feedback inter-agent communication is then investigated in this framework and it is indicated how the structure of interconnections and the amount of available information can affect stability performance.

ICRA Conference 2002 Conference Paper

Trajectory Design for Formations of Robots by Kinetic Energy Shaping

Calin Belta
Vijay Kumar 0001

We develop a method for generating smooth trajectories for a set of mobile robots. Given two end configurations, by tuning one parameter, the user can choose an interpolating trajectory from a continuum of curves varying from that corresponding to maintaining a rigid formation to motion of the robots toward each other The idea behind our method is to change the original constant kinetic energy metric in the configuration space and can be summarized into three steps. First, the energy of the motion as a rigid structure is decoupled from the energy of motion along directions that violate the rigid constraints. Next, the metric is "shaped" by assigning different weights to each term. Finally, geodesic flow is constructed for the modified metric. The optimal motions generated on the manifolds of rigid body displacements in 3D space or in plane, and the uniform rectilinear motion of each robot corresponding to a totally uncorrelated approach are particular cases of our general treatment.

ICRA Conference 2001 Conference Paper

An Architecture for Tightly Coupled Multi-Robot Cooperation

Luiz Chaimowicz
Thomas Sugar
Vijay Kumar 0001
Mario F. M. Campos

Proposes an architecture for tightly coupled multi-robot coordination that is well suited to cooperative manipulation tasks. At all times, a robot is identified as a leader, while the others are designated as followers. The assignment of roles and the coordination between the robots is guaranteed by communication protocols and control algorithms. The key feature is the flexibility that allows changes in leadership and assignment of roles during the execution of a task. We describe the experimental implementation and demonstration in a cooperative transportation task, in which two and three heterogeneous robots cooperate to carry a large object in an environment containing obstacles.

IROS Conference 2001 Conference Paper

Cooperative localization and control for multi-robot manipulation

John R. Spletzer
Aveek K. Das
Rafael Fierro
Camillo J. Taylor
Vijay Kumar 0001
Jim Ostrowski 0001

We describe a framework for coordinating multiple robots in cooperative manipulation tasks in which vision is used for establishing relative position and orientation and maintaining formation. The two key contributions are a cooperative scheme for localizing the robots based on visual imagery that is more robust than decentralized localization, and a set of control algorithms that allow the robots to maintain a prescribed formation (shape and size). The ability to maintain a prescribed formation allows the robots to "trap" objects in their midst, and to "flow" the formation to a desired position. We derive the cooperative localization and control algorithms and present experimental results that illustrate the implementation and the performance of these algorithms.

ICRA Conference 2001 Conference Paper

Coordination of Multiple Mobile Manipulators

Thomas Sugar
Jaydev P. Desai
Vijay Kumar 0001
Jim Ostrowski 0001

We present a novel modeling framework and control algorithms for multiple mobile manipulators cooperatively grasping and transporting an object. The planning and control tasks are decentralized, and the framework explicitly incorporates the protocols used to coordinate the robots in the team. The framework is flexible in the sense that it scales with the number of robots and controllers. Preliminary experimental results for teams of two-three robots are shown, while simulation results are used to illustrate the extensions of our approach to larger teams.

ICRA Conference 2001 Conference Paper

Dynamics and Control of Whole Arm Grasps

Peng Song 0005
Masahito Yashima
Vijay Kumar 0001

In this paper we consider the dynamics and control of whole arm grasping systems. We develop a control scheme that employs a minimal set of inputs to control the trajectory of the system while using the surplus inputs to control the interaction forces in order to maintain the unilateral constraints at both rolling and sliding contacts. Since the number of surplus inputs is less than the number of output force variables, we propose a controller that controls the critical contact force components. We emphasize the dynamic models and algorithms for computing contact forces, which are crucial to the development of the control algorithms. Finally, we show how compliant contact models and an integrated simulation approach are used to overcome the difficulties with uniqueness and existence of solutions. A planar whole arm manipulation system is used as an example to illustrate the basic ideas.

ICRA Conference 2001 Conference Paper

Efficient Dynamic Simulation of Robotic Systems with Hierarchy

Joel M. Esposito
Vijay Kumar 0001

In this paper multirate numerical integration techniques are introduced as a tool for simulating robotic systems. In contrast with traditional simulation techniques where a single global time step is used, multirate methods seek a gain in efficiency by using larger step sizes for the slow varying components and smaller step sizes for components with rapidly changing solutions. We argue that many robotic systems inherently possess different time scales, and therefore can benefit from multirate techniques. We have developed a multirate version of the popular Adams predictor-corrector methods, which has a variety of modern features. We present results on the accuracy, stability and efficiency of the algorithm along with simulation results.

ICRA Conference 2001 Conference Paper

Hybrid Control of Formations of Robots

Rafael Fierro
Aveek K. Das
Vijay Kumar 0001
Jim Ostrowski 0001

We describe a framework for controlling a group of nonholonomic mobile robots equipped with range sensors. The vehicles are required to follow a prescribed trajectory while maintaining a desired formation. By using the leader-following approach, we formulate the formation control problem as a hybrid (mode switching) control system. We then develop a decision module that allows the robots to automatically switch between continuous-state control laws to achieve a desired formation shape. The stability properties of the closed-loop hybrid system are studied using the Lyapunov theory. We do not use explicit communication between robots; instead we integrate optimal estimation techniques with nonlinear controllers. Simulation and experimental results verify the validity of our approach.

ICRA Conference 2001 Conference Paper

Motion Generation for Formations of Robots a Geometric Approach

Calin Belta
Vijay Kumar 0001

Develops a method for generating smooth trajectories for mobile robots in formation. The problem of trajectory generation is cast in terms of designing optimal curves on the Euclidean group, SE(3). Specifically, the method generates the trajectory that minimizes the total energy associated with the translations and rotations of the robots, while maintaining a rigid formation. When the mobile robots are nonholonomic, trajectories that allow rigid formations to be maintained must satisfy appropriate constraints. An efficient non-iterative algorithm to obtain near-optimal trajectories is described. Finally, the approach is illustrated with examples involving formations of aircrafts.

ICRA Conference 2001 Conference Paper

Real-Time Vision-Based Control of a Nonholonomic Mobile Robot

Aveek K. Das
Rafael Fierro
Vijay Kumar 0001
Ben Southall
John R. Spletzer
Camillo J. Taylor

This paper considers the problem of vision-based control of a nonholonomic mobile robot. We describe the design and implementation of real-time estimation and control algorithms on a car-like robot platform using a single omni-directional camera as a sensor without explicit use of odometry. We provide experimental results for each of these vision-based control objects. The algorithms are packaged as control modes and can be combined hierarchically to perform higher level tasks involving multiple robots.

ICRA Conference 2000 Conference Paper

Closed Loop Motion Plans for Mobile Robots

Joel M. Esposito
Vijay Kumar 0001

We discuss a game theoretic approach to the design of closed loop feedback laws to solve sensor based motion planning problems for mobile robots. Our approach provides a framework for dealing with environmental uncertainty, and, by explicitly accounting for the sensor dynamics, a formal way of combining exploratory and goal directed motions. We focus on methods of devising optimal feedback laws, using finite dimensional parametrizations, under the worst case uncertainty for several illustrative examples.

ICRA Conference 2000 Conference Paper

Dynamic Simulation for Grasping and Whole Arm Manipulation

Peng Song 0005
Masahito Yashima
Vijay Kumar 0001

We propose a novel approach to dynamic simulation of grasps that overcomes difficulties due to inconsistencies in the forward dynamics problem. The key idea in the paper is a minimalist model of the contact compliance and the use of this model in situations when the performance of the rigid body model is not satisfactory. Our general framework allows for online diagnostics that enable the automatic switching between models to maximize efficiency while avoiding ambiguous situations. We illustrate the basic ideas by simulating the dynamics of several whole arm grasps.

ICRA Conference 2000 Conference Paper

Metrics for Analysis and Optimization of Grasps and Fixtures

Thomas Sugar
Vijay Kumar 0001

We develop two quality measures for the analysis and optimization of grasps and fixtures. The first metric is a measure of stability and is obtained by analyzing the effect of small perturbations in the position of the grasped object or small disturbance forces on the grasped object The second metric is a measure of robustness to errors in the positions of the fixture elements. In addition, we explore the effect that the stiffness of the fixture elements has on these two quality measures and propose a design optimization approach to maximize stability and robustness to errors.

ICRA Conference 2000 Conference Paper

Robotic Grasping and Contact: A Review

Antonio Bicchi
Vijay Kumar 0001

In this paper, we survey the field of robotic grasping and the work that has been done in this area over the last two decades, with a slight bias toward the development of the theoretical framework and analytical results in this area.

ICRA Conference 1999 Conference Paper

Control of Changes in Formation for a Team of Mobile Robots

Jaydev P. Desai
Vijay Kumar 0001
Jim Ostrowski 0001

Addresses the control of a team of robots navigating in a terrain with obstacles while maintaining a desired formation and changing formations when required. We model the team as a triple consisting of a group element that describes the gross motion of the team, a set of shape variables that describe the relative positions of robots, and a control graph that describes the behaviors of the robots in the formation. We assume that a lead robot is equipped with the appropriate sensors and plans the gross motion (path) for the team. This path is derived from optimal control theory. All other robots are coordinated by continuous controllers that are prescribed by the control graph. Our framework allows us to enumerate the number of control graphs and the possible transitions between them. Further, we describe an algorithm that allows the team of robots to move between any two formations, while avoiding obstacles. We illustrate the methodology with examples involving teams of 5 and 6 robots in the presence of obstacles.

ICRA Conference 1999 Conference Paper

Multiple Cooperating Mobile Manipulators

Thomas Sugar
Vijay Kumar 0001

The main goal of the paper is to present a framework and basic control algorithms for coordinating a small team of robots in tasks that involve grasping large objects and transporting them in a two-dimensional environment with obstacles. Each robot consists of a mobile platform and an arm. Some arms are passive while others are active. Our previous work (1998) addressed the manipulator design, the architecture, and the communication between robots. In this paper we address the control of the manipulators, the coordination between the platforms for a team of two or three cooperating robots, and the handling of flexible large objects.

ICRA Conference 1998 Conference Paper

Controlling Formations of Multiple Mobile Robots

Jaydev P. Desai
Jim Ostrowski 0001
Vijay Kumar 0001

We investigate feedback laws used to control multiple robots moving together in a formation. We propose a method for controlling formations that uses only local sensor-based information, in a leader-follower motion. We use methods of feedback linearization to exponentially stabilize the relative distance and orientation of the follower, and show that the zero dynamics of the system are also (asymptotically) stable. We demonstrate in simulation the use of these algorithms to control six robots moving around an obstacle. These types of control laws can be used to control arbitrarily large numbers of robots moving in very general types of formations.

ICRA Conference 1998 Conference Paper

Decentralized Control of Cooperating Mobile Manipulators

Thomas Sugar
Vijay Kumar 0001

This paper addresses the design and control of autonomous mobile platforms manipulating and transporting a grasped object. Each platform is equipped with a manipulator arm. In order to hold an object in a stable grasp, the arms have to apply and maintain appropriate contact forces to ensure force closure. We describe an architecture and a real time control system that allow the coordinated control of multiple, heterogeneous, mobile manipulators. In this architecture, a lead robot plans, based on available sensory information, and follows a suitable trajectory. The other robots follow a desired formation with respect to the leader while maintaining a stable grasp. There are three main contributions in this paper. We present the real time control system and our approach to rapidly prototype the control code. We describe a novel, forklift-like arm and a control scheme that allows us to easily control the Cartesian stiffness or impedance. Finally, we describe the details of an experimental testbed and results that demonstrate the robustness of the control scheme.

ICRA Conference 1998 Conference Paper

Generalized Stability of Compliant Grasps

Herman Bruyninckx
Sabine Demey
Vijay Kumar 0001

We develop a geometric framework for the stability analysis of multifingered grasps and propose a measure of grasp stability for arbitrary perturbations and loading conditions. The measure requires a choice of metric on the group of rigid body displacements. We show that although the stability of a grasp itself does not depend on the choice of metric, comparison of the stability of different grasps depends on the metric. Finally, we provide some insight into the choice of metrics for stability analysis.

ICRA Conference 1998 Conference Paper

Motion Planning with Uncertainty

Hong Zhang 0015
Vijay Kumar 0001
Jim Ostrowski 0001

We present a general framework for motion planning of robots in the presence of obstacles and other robots. We use variational calculus and optimization to find optimal open loop and closed loop plans in the presence of uncertainty. The plans are based on world models with set-valued uncertainty associated with the positions and shape of the obstacles. The open loop plans are generated by an efficient method that allows successive refinements of a nominal motion plan and accommodates finer levels of granularity as additional information becomes available. The closed loop plans are control policies that are based on given sensor models. The optimal plan is a control policy that performs the best in the worst-case situation. We discuss how the open loop and closed loop plans can be viewed as optimal strategies in the framework of two-person, zero-sum, non-cooperative games.

ICRA Conference 1998 Conference Paper

Two Methods for Interpolating Rigid Body Motions

Milos Zefran
Vijay Kumar 0001

This paper investigates methods for computing a smooth motion that interpolates a given set of positions and orientations of a rigid body. To make the interpolation independent of the representation of the motion, we use the coordinate-free framework of differential geometry. Inertial and body-fixed reference frames must be chosen to describe the position and orientation of the rigid body. We show that trajectories that are independent of the choice of these frames can be obtained by using the exponential map. Since these trajectories may exhibit rapid changes in velocity or its higher derivatives, a method for finding the maximally smooth interpolating curve is developed. Trajectories computed by both methods are compared on an example.

ICRA Conference 1997 Conference Paper

Affine connections for the Cartesian stiffness matrix

Milos Zefran
Vijay Kumar 0001

We study the 6/spl times/6 Cartesian stiffness matrix. We show that the stiffness of a rigid body subjected to conservative forces and moments is described by a (0, 2) tensor which is the Hessian of the potential function. The key observation of the paper is that since the Hessian depends on the choice of an affine connection in the task space, so will the Cartesian stiffness matrix. Further, the symmetry of the Hessian and thus of the stiffness matrix depends on the symmetry of the connection. The connection that is implicit in the definition of the Cartesian stiffness matrix through the joint stiffness matrix (Salisbury, 1980) is made explicit and shown to be symmetric. In contrast, the direct definition of the Cartesian stiffness matrix in Griffis (1993), Ciblak and Lipkin (1994) and Howard et al. (1996) is shown to be derived from an asymmetric connection. A numerical example is provided to illustrate the main ideas of the paper.

ICRA Conference 1997 Conference Paper

Nonholonomic motion planning for multiple mobile manipulators

Jaydev P. Desai
Vijay Kumar 0001

We address the problem of motion planning for nonholonomic cooperating mobile robots manipulating and transporting objects while holding them in a stable grasp. We present a general approach based on the calculus of variations that allows us to obtain optimal trajectories and actuator forces/torques for any manoeuvre in the presence of obstacles. In addition, geometric constraints such as joint limits, kinematic constraints such as nonholonomic velocity constraints and dynamic constraints can be easily incorporated into the planning scheme. The application of the method is illustrated by computing motion plans for several examples.

ICRA Conference 1997 Conference Paper

Optimal gait selection for nonholonomic locomotion systems

Jim Ostrowski 0001
Jaydev P. Desai
Vijay Kumar 0001

This paper addresses the optimal control and selection of gaits in a class of nonholonomic locomotion systems that exhibit group symmetries. We study optimal gaits for the snakeboard, a representative example of this class of systems. We employ Lagrangian reduction techniques to simplify the optimal control problem, and describe a general framework and an algorithm to obtain numerical solutions to this problem. This paper represents an initial study in using optimal control techniques to study optimality of gaits and issues involving gait transitions. The general framework provided in this paper can easily be applied to other examples of biological and robotic locomotion.

IROS Conference 1997 Conference Paper

Two-arm manipulation tasks with friction assisted grasping

Jaydev P. Desai
Milos Zefran
Vijay Kumar 0001

This paper studies human dual arm manipulation tasks and develops a computational model that predicts the trajectories and the force distribution for the coordination of two arms moving an object between two given positions and orientations in a horizontal plane. Our ultimate goal is to understand the dynamics of dual arm coordination in order to develop better robot control algorithms. Our computational model is based on the hypothesis proposed by Uno et al. (1989) who suggest that human movements minimize the integral of the norm of the rate of change of actuator torques. We compare the experimental trajectories and force distributions with this computational model. The first important observation is that the trajectories show a significant degree of repeatability across trials and across subjects. Next, we observe that the trajectories in the sagittal and frontal plane are characterized by asymmetric features that are hard to model using such integral cost functions. Finally, we show that the internal forces play an important role in trajectory generation. While these are repeatable across trials, they vary significantly from subject to subject.

ICRA Conference 1996 Conference Paper

Motion planning for multiple mobile manipulators

Jaydev P. Desai
Chau-Chang Wang
Milos Zefran
Vijay Kumar 0001

We address the motion planning for "fixtureless" material-handling with multiple manipulators on nonholonomic carts. The mobile manipulators possess the ability to manipulate and transport objects while holding them in a stable grasp. We present a general approach that allows generation of optimal trajectories and actuator inputs for any given maneuver. Constraints such as limitations on the turning radii of the mobile manipulators or bounds on their separation can be easily incorporated into the planning scheme. Numerical solutions for several maneuvers including abrupt turns, parallel parking in cluttered environments and changes in formation are computed. Finally, we present experimental results with two mobile manipulators.

ICRA Conference 1996 Conference Paper

Motion planning of walking robots in environments with uncertainty

Chun-Hung Chen
Vijay Kumar 0001

Presents a general approach for coordinating the legs of a multi-legged statically stable walking machine on an uneven terrain. The approach yields an optimized motion plan for several "body-lengths" that allows us to select footholds and sequence the legs of the walking machine. We assume that a terrain map is available but that this map may be characterized by uncertainty. We also assume the optimality of the motion plan can be measured by a suitable metric. The method of ordinal optimization is used to find a motion plan that is guaranteed to be in a desired percentile with a given confidence level. Depending on the available computational resources we can improve our confidence level and/or get closer to the optimal plan. Finally, our approach allows us to trade off speed with safety, and speed with optimality.

ICRA Conference 1996 Conference Paper

Planning of smooth motions on SE(3)

Milos Zefran
Vijay Kumar 0001

This paper addresses the general problem of generating smooth trajectories between an initial and a final position and orientation. A functional depending an velocity and its higher derivatives involving a left invariant Riemannian metric on SE(3) is used to measure the smoothness of a trajectory. The problem of determining a smooth trajectory between two points is formulated as a variational problem on SE(3). The authors derive necessary conditions for the shortest distance and minimum jerk trajectories and solve the resulting two-point boundary value problem.

IROS Conference 1995 Conference Paper

Cooperative material handling by human and robotic agents: module development and system synthesis

Julie A. Adams
Ruzena Bajcsy
Jana Kosecka
Vijay Kumar 0001
Robert Mandelbaum
Max Mintz
Richard P. Paul
Curtis Wang

Presents a collaborative effort to design and implement a cooperative material handling system by a small team of human and robotic agents in an unstructured indoor environment. The authors' approach makes fundamental use of the human agents' expertise for aspects of task planning, task monitoring, and error recovery. The authors' system is neither fully autonomous nor fully teleoperated. It is designed to make effective use of the human's abilities within the present state of the art of autonomous systems. The authors' robotic agents refer to systems which are each equipped with at least one sensing modality and which possess some capability for self-orientation and/or mobility. The authors' robotic agents are not required to be homogeneous with respect to either capabilities or function. The authors' research stresses both paradigms and testbed experimentation. Theory issues include the requisite coordination principles and techniques which are fundamental to a cooperative multiagent system's basic functioning. The authors have constructed an experimental distributed multiagent-architecture testbed facility. The required modular components of this testbed are currently operational and have been tested individually. The authors' current research focuses on the agents' integration in a scenario for cooperative material handling.

ICRA Conference 1995 Conference Paper

Modeling and Analysis of the Compliance and Stability of Enveloping Graspa

W. Stamps Howard
Vijay Kumar 0001

We address the stability and performance of enveloping grasps. An enveloping grasp is defined in which robot fingers or arms grasp an object by making contact at points other than just the fingertips or end-effectors. Previous work has demonstrated that enveloping grasps are more robust. However their analysis is difficult because they are usually statically indeterminate and overconstrained. In this paper we develop a method for modeling the stiffness of a grasp. It is based on a model of compliance at each contact and a model of the joint compliance at each joint. It incorporates the kinematics of three-dimensional contact in addition to the kinematics of the robot arm/finger. In particular the methodology is used to show that force closure does not necessarily imply stability.

ICRA Conference 1995 Conference Paper

Optimal Control of Systems with Unilateral Constraints

Milos Zefran
Vijay Kumar 0001

Problems in robotics and biomechanics such as trajectory planning or resolution of redundancy can be effectively solved using optimal control. Such systems are often subject to unilateral constraints. Examples include tasks involving contacts (e. g. , walking, running, multifingered or multiarm manipulation), and other tasks that may not involve contacts but in which the system state or the inputs must satisfy inequality conditions (e. g. , limits on actuator forces). This paper shows how problems of optimal control in robotics that involve unilateral constraints can be efficiently solved by first formulating the constrained optimal control problem as an unconstrained problem of the calculus of variations and then solving it using an integral formulation. This method has several advantages over the Pontryagin minimum principle which is traditionally employed to solve such problems. An example of two-arm manipulation with inequality constraints due to Coulomb friction is used to demonstrate the formulation of the problem and the algorithms.

ICRA Conference 1995 Conference Paper

The Performance of a Repeatable Control Scheme for Redundant Robots

Chau-Chang Wang
Vijay Kumar 0001

This paper addresses the determination of joint motions for a given end effector displacement in kinematically redundant manipulators. It is well known that velocity control schemes, such as pseudoinverse control, do not guarantee repeatability. But it is possible to obtain repeatable solutions by controlling the manipulator joints so that they behave like elastic couplings. The joint configuration adopted by the manipulator at a given point in task space is one which minimizes the potential energy of the system and is unique. In this paper, we investigate the existence and uniqueness of solutions (joint displacements) for a given end effector displacement. We identify singularities in the algorithm and their physical significance. Finally, we show that it is important to consider the stability of the manipulator at a given configuration and discuss practical considerations in implementing such algorithms.

IROS Conference 1995 Conference Paper

Two-arm manipulation: what can we learn by studying humans?

Milos Zefran
Vijay Kumar 0001
Jaydev P. Desai
Ealan Henis

This paper addresses determination of trajectories and force distribution for cooperative manipulation with two arms through optimizing an integral cost function that depends an the actuator forces. We compare the calculated trajectories with the measurements on human subjects performing planar manipulation tasks. Our findings suggest that the trajectories and forces used by humans can be predicted by minimizing the integral of the rate of change of actuator torques over the trajectory. Good match is shown for a class of manipulation tasks in which the person-to-person variability is small. The theoretical foundation for computing the optimal solutions is briefly presented and the advantages of using such schemes for robotic systems are discussed.

ICRA Conference 1994 Conference Paper

An Adaptive Mobility System for the Disabled

Parris S. Wellman
Venkat Krovi
Vijay Kumar 0001

A proof-of-concept prototype walking chair for the disabled is proposed with the objective of demonstrating the feasibility of a completely new approach to mobility. Our prototype system consists of a chair equipped with wheels and legs and is capable of walking on uneven terrain and circumventing obstacles. The important design considerations, the system design and an experimental prototype of a chair, are discussed. Redundancy in actuation enables the online optimization of tractive forces which enhances the adaptability of the system. >

ICRA Conference 1994 Conference Paper

Control of a Single Robot in a Decentralized Multi-Robot System

Nilanjan Sarkar
Xiaoping Yun
Vijay Kumar 0001

In a decentralized multi-robot system, each robot is controlled by an independent controller and the information obtained by each robot through its proprioceptive sensing devices is not shared with other robots. If the coordination of multiple robots in a cooperative manipulation task is to be accomplished, each robot must be able to exhibit controlled interaction with other robots and objects with minimal information. In this paper, the authors address the problem of maintaining rolling contact between a robot arm and an external, moving object. The authors assume that a nominal model of the motion of the moving object is available. The dynamic system that characterizes the motion of the robot arm and the moving object is acatastatic and nonholonomic. The authors design a nonlinear feedback for such a system that successfully maintains rolling contact. Simulation results are presented to demonstrate the effectiveness of the system. >

ICRA Conference 1994 Conference Paper

Optimal Trajectories and Force Distribution for Cooperating Arms

Milos Zefran
Vijay Kumar 0001
Xiaoping Yun

The optimization of trajectories and actuator torques for a dual arm manipulation system is considered. Given the starting and final configurations, we find the trajectories that minimize: (a) the integral of the norm of the vector of derivatives of the actuator forces; and (b) the integral of the norm of the actuator forces. In this way both kinematic and actuator redundancy are resolved. The optimization problem reduces to solving a two-point boundary valve problem for coupled, nonlinear differential equations. The effect of different parameters such as preload and inertia are investigated and the results are compared with those obtained using other well-known cost functions. >

ICRA Conference 1994 Conference Paper

Stability of Planar Grasps

W. Stamps Howard
Vijay Kumar 0001

It is well established that all form closed grasps are stable. However, not all stable grasps are form closed, including many common and easily obtainable grasps. This is especially true in whole arm manipulators and multi-fingered hands where, in many cases, it is difficult or impossible to effect a form closed grasp. Thus it is necessary to establish a general framework for the determination of stability for any equilibrium grasp. In order to analyze the stability of grasps with multiple contacts, we first model the compliance at each contact. We develop expressions for the changes in contact forces as a function of the rigid body relative motion between the fingers and a grasped planar object. The stability of a grasp is shown to depend on the local curvature of the contacting bodies as well as the magnitude and arrangement of the contact forces. We then derive results providing simple criteria to determine the stability of almost any grasped planar object, including the special but important limiting case of rigid bodies where the contact compliance is zero. >

IROS Conference 1993 Conference Paper

Control of a single robot in a cooperative multi-robot framework

Nilanjan Sarkar
Xiaoping Yun
Vijay Kumar 0001

In a versatile, intelligent, multirobot system, each robot is controlled by independent controllers and information obtained by a robot through proprioceptive sensing (joint position, velocity, torques etc.) is not shared with other robots. If the coordination of robots in a cooperative manipulation task is to be accomplished, each robot must be able to exhibit controlled interaction with other robots and objects with minimal information. The authors address the problem of maintaining rolling contact between am actively controlled mechanical linkage and an external moving surface, assuming that a nominal model of the gross motion of the moving surface is available. Thus the control problem involves a dynamic system that is acatastic and nonholonomic. Nonlinear feedback is used to decouple and linearize the time-varying system. Examples and results from computer simulations are used to show that the normal and tangential forces as well as the relative rolling motion can be controlled satisfactorily.

ICRA Conference 1992 Conference Paper

A compact inverse velocity solution for redundant robots

Vijay Kumar 0001

A compact inverse rate kinematics solution for serial-chain redundant n-jointed robots is presented. The serial-chain inverse rate kinematics problem is dual to the inverse statics problem in in-parallel systems such as cooperating robots, in which the task of distributing the load wrench between multiple subsystems is typically under-specified. Therefore, the method proposed earlier in J. H. Kim and V. Kumar (1991) and V. Kumar (1990) for obtaining the wrench distribution in multirobot systems can be applied to redundant serial-chain manipulators. The solution thus obtained is one in which a weighted-norm of the joint rates vector is minimized. The advantages of this method are the ability to accommodate any type of joint and the availability of analytical, closed-form solutions for the joint rates. A geometric interpretation of the solution is provided using screw system theory. >

ICRA Conference 1992 Conference Paper

Control of multiple arms with rolling constraints

Xiaoping Yun
Vijay Kumar 0001
Nilanjan Sarkar
Eric Paljug

The authors present a unified formulation for the control problem of multiple arm systems which accommodates both holonomic and nonholonomic constraints. Several unique control properties of nonholonomic systems are discussed. Several useful results regarding input-output linearization and the zero dynamics in such systems are proved. The analysis and controller design are discussed for multi-arm systems. Results from computer simulations are presented to demonstrate the control algorithms. Simulation results illustrate that rolling and sliding can be effectively controlled. >

ICRA Conference 1992 Conference Paper

Some important considerations in force control implementation

Eric Paljug
Thomas Sugar
Vijay Kumar 0001
Xiaoping Yun

The authors address force control in overconstrained dynamic systems with special emphasis on robot control. Previous approaches to force control are studied, and many of these are shown to be unsuitable for dynamic force control. Practical and theoretical considerations for designing force control algorithms are discussed. Experimental and simulation results that validate the theoretical findings are presented for a single-degree-of-freedom pneumatic force controller. >

ICRA Conference 1991 Conference Paper

Control of contact conditions for manipulation with multiple robotic systems

Vijay Kumar 0001
Xiaoping Yun
Eric Paljug
Nilanjan Sarkar

An object much larger than a robot end-effector can not be rigidly grasped. Instead, it is necessary to support the object with multiple effectors. The effectors need not be grippers-they may be surfaces on the arm(s). Multiple arm manipulation without rigid grasps is investigated. The interaction between a robot and the object is characterized by unilateral constraints. The closed chain topology, redundancy in actuation, and the nonlinear coupled equations of motion make the problem formidable. The approach used is to utilize a minimal set of inputs to control the trajectory. The surplus inputs are used to control the contact conditions (rolling, sliding, etc.). >

ICRA Conference 1991 Conference Paper

Design and control of a 3-DOF in-parallel actuated manipulator

George H. Pfreundschuh
Vijay Kumar 0001
Thomas Sugar

The mechanics, design, control, and experimental results for a 3-DOF (degree-of-freedom) in-parallel, pneumatically actuated manipulator are presented. The manipulator consists of two platforms connected by three serial chains. It is naturally compliant in translation along the approach direction as well as in rotations about axes perpendicular to the approach direction, while it is stiff in other directions. The kinematic design allows the three relative DOF between the two plates of the manipulator to accommodate uncertainties and sustain impacts while contacting and interacting with unknown environments. Experimental results show that, with a sampling rate of 100 Hz, the position control bandwidth of a single actuator servo system is 11 Hz, while the force control bandwidth is 10 Hz. These figures also reflect the bandwidth and the performance of the complete system. >

ICRA Conference 1989 Conference Paper

Kinematics and control of redundantly actuated closed chains

John F. Gardner
Vijay Kumar 0001
J. H. Ho

The instantaneous kinematics are discussed of a hybrid manipulation system that combines the traditional serial chain geometry with parallelism in actuation. Such a system is characterized by closed chains in the structure and redundancy in actuation. This redundancy is shown to be dual to the kinematic redundancy in serial chain robot manipulators. The presence of redundancy in the system allows the specification of force set-points that will ensure an optimal load distribution for force control or hybrid control schemes. In addition, the singularities in the inverse kinematics and statics equations, which are typical of robotic systems with closed chains, are analyzed. Coordination algorithms for the computation of optimal force distribution that minimize joint torques while avoiding singularities with reasonable computation efficiency are the authors' main focus. In particular, a planar dual-arm manipulation system is used as an example and is analyzed in some detail. >