Arrow Research search

Author name cluster

George J. Pappas

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

88 papers
2 author rows

Possible papers

88

ICML Conference 2025 Conference Paper

Adversarial Reasoning at Jailbreaking Time

  • Mahdi Sabbaghi
  • Paul Kassianik
  • George J. Pappas
  • Amin Karbasi
  • Seyed Hamed Hassani

As large language models (LLMs) are becoming more capable and widespread, the study of their failure cases is becoming increasingly important. Recent advances in standardizing, measuring, and scaling test-time compute suggest new methodologies for optimizing models to achieve high performance on hard tasks. In this paper, we apply these advances to the task of model jailbreaking: eliciting harmful responses from aligned LLMs. We develop an adversarial reasoning approach to automatic jailbreaking that leverages a loss signal to guide the test-time compute, achieving SOTA attack success rates against many aligned LLMs, even those that aim to trade inference-time compute for adversarial robustness. Our approach introduces a new paradigm in understanding LLM vulnerabilities, laying the foundation for the development of more robust and trustworthy AI systems.

Details

TMLR Journal 2025 Journal Article

Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation

  • Yutong He
  • Alexander Robey
  • Naoki Murata
  • Yiding Jiang
  • Joshua Nathaniel Williams
  • George J. Pappas
  • Hamed Hassani
  • Yuki Mitsufuji

Prompt engineering is an effective but labor-intensive way to control text-to-image (T2I) generative models. Its time-intensive nature and complexity have spurred the development of algorithms for automated prompt generation. However, these methods often struggle with transferability across T2I models, require white-box access to the underlying model, or produce non-intuitive prompts. In this work, we introduce PRISM, an algorithm that automatically produces human-interpretable and transferable prompts that can effectively generate desired concepts given only black-box access to T2I models. Inspired by large language model (LLM) jailbreaking, PRISM leverages the in-context learning ability of LLMs to iteratively refine the candidate prompt distribution built upon the reference images. Our experiments demonstrate the versatility and effectiveness of PRISM in generating accurate prompts for objects, styles, and images across multiple T2I models, including Stable Diffusion, DALL-E, and Midjourney.

PDF Details

NeurIPS Conference 2025 Conference Paper

Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization

  • Sunay Joshi
  • Shayan Kiyani
  • George J. Pappas
  • Edgar Dobriban
  • Hamed Hassani

We consider the problem of conformal prediction under covariate shift. Given labeled data from a source domain and unlabeled data from a covariate shifted target domain, we seek to construct prediction sets with valid marginal coverage in the target domain. Most existing methods require estimating the unknown likelihood ratio function, which can be prohibitive for high-dimensional data such as images. To address this challenge, we introduce the likelihood ratio regularized quantile regression (LR-QR) algorithm, which combines the pinball loss with a novel choice of regularization in order to construct a threshold function without directly estimating the unknown likelihood ratio. We show that the LR-QR method has coverage at the desired level in the target domain, up to a small error term that we can control. Our proofs draw on a novel analysis of coverage via stability bounds from learning theory. Our experiments demonstrate that the LR-QR algorithm outperforms existing methods on high-dimensional prediction tasks, including a regression task for the Communities and Crime dataset, an image classification task from the WILDS repository, and an LLM question-answering task on the MMLU benchmark.

PDF Details

NeurIPS Conference 2025 Conference Paper

Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models

  • Sima Noorani
  • Shayan Kiyani
  • George J. Pappas
  • Hamed Hassani

Uncertainty quantification (UQ) is essential for safe deployment of generative AI models such as large language models (LLMs), especially in high-stakes applications. Conformal prediction (CP) offers a principled uncertainty quantification framework, but classical methods focus on regression and classification, relying on geometric distances or softmax scores--tools that presuppose structured outputs. We depart from this paradigm by studying CP in a query-only setting, where prediction sets must be constructed solely from finite queries to a black-box generative model, introducing a new trade-off between coverage, test-time query budget, and informativeness. We introduce Conformal Prediction with Query Oracle (CPQ), a framework characterizing the optimal interplay between these objectives. Our finite-sample algorithm is built on two core principles: one governs the optimal query policy, and the other defines the optimal mapping from queried samples to prediction sets. Remarkably, both are rooted in the classical missing mass problem in statistics. Specifically, the optimal query policy depends on the rate of decay--or the derivative--of the missing mass, for which we develop a novel estimator. Meanwhile, the optimal mapping hinges on the missing mass itself, which we estimate using Good-Turing estimators. We then turn our focus to implementing our method for language models, particularly in open-ended LLM tasks involving question answering, multi-step reasoning, and structured information extraction, where outputs are vast, variable, and often under-specified. Fine-grained experiments on three real-world open-ended tasks and two LLMs, show CPQ's applicability to any black-box LLM and highlight: (1) individual contribution of each principle to CPQ’s performance, and (2) CPQ's ability to yield significantly more informative prediction sets than existing conformal methods for language uncertainty quantification.

PDF Details

ICLR Conference 2025 Conference Paper

CViT: Continuous Vision Transformer for Operator Learning

  • Sifan Wang
  • Jacob H. Seidman
  • Shyam Sankaran
  • Hanwen Wang
  • George J. Pappas
  • Paris Perdikaris

Operator learning, which aims to approximate maps between infinite-dimensional function spaces, is an important area in scientific machine learning with applications across various physical domains. Here we introduce the Continuous Vision Transformer (CViT), a novel neural operator architecture that leverages advances in computer vision to address challenges in learning complex physical systems. CViT combines a vision transformer encoder, a novel grid-based coordinate embedding, and a query-wise cross-attention mechanism to effectively capture multi-scale dependencies. This design allows for flexible output representations and consistent evaluation at arbitrary resolutions. We demonstrate CViT's effectiveness across a diverse range of partial differential equation (PDE) systems, including fluid dynamics, climate modeling, and reaction-diffusion processes. Our comprehensive experiments show that CViT achieves state-of-the-art performance on multiple benchmarks, often surpassing larger foundation models, even without extensive pretraining and roll-out fine-tuning. Taken together, CViT exhibits robust handling of discontinuous solutions, multi-scale features, and intricate spatio-temporal dynamics. Our contributions can be viewed as a significant step towards adapting advanced computer vision architectures for building more flexible and accurate machine learning models in the physical sciences.

Details

ICML Conference 2025 Conference Paper

Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents

  • Shayan Kiyani
  • George J. Pappas
  • Aaron Roth 0001
  • Seyed Hamed Hassani

A fundamental question in data-driven decision making is how to quantify the uncertainty of predictions to inform risk-sensitive downstream actions, as often required in domains such as medicine. We develop a decision-theoretic foundation linking prediction sets to risk-averse decision-making, addressing three questions: (1) What is the correct notion of uncertainty quantification for risk-averse decision makers? We prove that prediction sets are optimal for decision makers who wish to optimize their value at risk. (2) What is the optimal policy that a risk averse decision maker should use to map prediction sets to actions? We show that a simple max-min decision policy is optimal for risk-averse decision makers. Finally, (3) How can we derive prediction sets that are optimal for such decision makers? We provide an exact characterization in the population regime and a distribution free finite-sample construction. These insights leads to Risk-Averse Calibration (RAC), a principled algorithm that is both practical —exploiting black-box predictions to enhance downstream utility—and safe —adhering to user-defined risk thresholds. We experimentally demonstrate RAC’s advantages in medical diagnosis and recommendation systems, showing that it substantially improves the trade-off between safety and utility, delivering higher utility than existing methods while avoiding critical errors.

Details

IROS Conference 2025 Conference Paper

Deep Equivariant Multi-Agent Control Barrier Functions

  • Nikolaos Bousias
  • Lars Lindemann
  • George J. Pappas

With multi-agent systems increasingly deployed autonomously at scale in complex environments, ensuring safety of the data-driven policies is critical. Control Barrier Functions have emerged as an effective tool for enforcing safety constraints, yet existing learning-based methods often lack in scalability, generalization and sampling efficiency as they overlook inherent geometric structures of the system. To address this gap, we introduce symmetries-infused distributed CBFs, enforcing the satisfaction of intrinsic symmetries on learnable graph-based safety certificates. We theoretically motivate the need for equivariant parametrization of CBFs and policies, and propose a simple, yet efficient and adaptable methodology for constructing such equivariant group-modular networks via the compatible group actions. This approach encodes safety constraints in a distributed data-efficient manner, enabling zero-shot generalization to larger and denser swarms. Through extensive simulations on multi-robot navigation tasks, we demonstrate that our method outperforms state-of-the-art baselines in terms of safety, scalability, and task success rates, highlighting the importance of embedding symmetries in safe distributed neural policies.

Details

ICRA Conference 2025 Conference Paper

Flying Quadrotors in Tight Formations Using Learning-Based Model Predictive Control

  • Kong Yao Chee
  • Pei-An Hsieh
  • George J. Pappas
  • M. Ani Hsieh

Flying quadrotors in tight formations is a challenging problem. It is known that in the near-field airflow of a quadrotor, the aerodynamic effects induced by the propellers are complex and difficult to characterize. Although machine learning tools can potentially be used to derive models that capture these effects, these data-driven approaches can be sample inefficient and the resulting models often do not generalize as well as their first-principles counterparts. In this work, we propose a framework that combines the benefits of first-principles modeling and data-driven approaches to construct an accurate and sample efficient representation of the complex aerodynamic effects resulting from quadrotors flying in formation. The data-driven component within our model is lightweight, making it amenable for optimization-based control design. Through simulations and physical experiments, we show that incorporating the model into a novel learning-based nonlinear model predictive control (MPC) framework results in substantial performance improvements in terms of trajectory tracking and disturbance rejection. In particular, our framework significantly outperforms nominal MPC in physical experiments, achieving a 40. 1% improvement in the average trajectory tracking errors and a 57. 5% reduction in the maximum vertical separation errors. Our framework also achieves exceptional sample efficiency, using only a total of 46 seconds of flight data for training across both simulations and physical experiments. Furthermore, with our proposed framework, the quadrotors achieve an exceptionally tight formation, flying with an average separation of less than 1. 5 body lengths throughout the flight.

Details

ICRA Conference 2025 Conference Paper

Jailbreaking LLM-Controlled Robots

  • Alexander Robey
  • Zachary Ravichandran
  • Vijay Kumar 0001
  • Seyed Hamed Hassani
  • George J. Pappas

The recent introduction of large language models (LLMs) has revolutionized the field of robotics by enabling contextual reasoning and intuitive human-robot interaction in domains as varied as manipulation, locomotion, and self-driving vehicles. When viewed as a stand-alone technology, LLMs are known to be vulnerable to jailbreaking attacks, wherein mali-cious prompters elicit harmful text by bypassing LLM safety guardrails. To assess the risks of deploying LLMs in robotics, in this paper, we introduce ROBOPAIR, the first algorithm designed to jailbreak LLM-controlled robots. Unlike existing, textual attacks on LLM chatbots, Robopairelicits harmful physical actions from LLM-controlled robots, a phenomenon we experimentally demonstrate in three scenarios: (i) a white-box setting, wherein the attacker has full access to the NVID IA Dolphins self-driving LLM, (ii) a gray-box setting, wherein the attacker has partial access to a Clearpath Robotics Jackal UGV robot equipped with a GPT-40 planner, and (iii) a black-box setting, wherein the attacker has only query access to the GPT-3. 5-integrated Unitree Robotics Go2robot dog. In each scenario and across three new datasets of harmful robotic actions, we demonstrate that ROBOPAIR, as well as several static baselines, finds jailbreaks quickly and effectively, often achieving 100 % attack success rates. Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world. Indeed, our results on the U nitree G02represent the first successful jailbreak of a deployed commercial robotic system. Addressing this emerging vulnerability is critical for ensuring the safe deployment of LLMs in robotics. Additional media is available at: https://robopair.org.

Details

TMLR Journal 2025 Journal Article

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

  • Alexander Robey
  • Eric Wong
  • Hamed Hassani
  • George J. Pappas

Despite efforts to align large language models (LLMs) with human intentions, widely-used LLMs such as GPT, Llama, and Claude are susceptible to jailbreaking attacks, wherein an adversary fools a targeted LLM into generating objectionable content. To address this vulnerability, we propose SmoothLLM, an algorithm designed to mitigate jailbreaking attacks. Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs. Across a range of popular LLMs, SmoothLLM offers improved robustness against the GCG, PAIR, RandomSearch, and AmpleGCG jailbreaks. SmoothLLM is also resistant against adaptive GCG attacks, exhibits a small, though non-negligible trade-off between robustness and nominal performance, and is compatible with any LLM.

PDF Details

ICRA Conference 2025 Conference Paper

SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environments

  • Zachary Ravichandran
  • Varun Murali
  • Mariliza Tzes
  • George J. Pappas
  • Vijay Kumar 0001

As robots become increasingly capable, users will want to describe high-level missions and have robots infer the relevant details. Because pre-built maps are difficult to obtain in many realistic settings, accomplishing such missions will require the robot to map and plan online. While many semantic planning methods operate online, they are typically designed for well specified missions such as object search or exploration. Recently, Large Language Models (LLMs) have demonstrated powerful contextual reasoning abilities over a range of robotic tasks described in natural language. However, existing LLM-enabled planners typically do not consider online planning or complex missions; rather, relevant subtasks and semantics are provided by a pre-built map or a user. We address these limitations via SPINE, an online planner for missions with incomplete mission specifications provided in natural language. The planner uses an LLM to reason about subtasks implied by the mission specification and then realizes these subtasks in a receding horizon framework. Tasks are automatically validated for safety and refined online with new map observations. We evaluate SPINE in simulation and real-world settings with missions that require multiple steps of semantic reasoning and exploration in cluttered outdoor environments of over 20, 000m 2. Compared to baselines that use existing LLM-enabled planning approaches, our method is over twice as efficient in terms of time and distance, requires less user interactions, and does not require a full map. Additional resources are provided at https://zacravichandran.github.io/SPINE.

Details

NeurIPS Conference 2025 Conference Paper

Uncertainty-Calibrated Prediction of Randomly-Timed Biomarker Trajectories with Conformal Bands

  • Vasiliki Tassopoulou
  • Charis Stamouli
  • Haochang Shou
  • George J. Pappas
  • Christos Davatzikos

We introduce a novel conformal prediction framework for constructing conformal prediction bands with high probability around biomarker trajectories observed at subject-specific, randomly-timed follow-up visits. Existing conformal methods typically assume fixed time grids, limiting their applicability in longitudinal clinical studies. Our approach addresses this limitation by defining a time-varying nonconformity score that normalizes prediction errors using model-derived uncertainty estimates, enabling conformal inference at arbitrary time points. We evaluate our method on two well-established brain biomarkers—hippocampal and ventricular volume—using a range of standard and state-of-the-art predictors. Across models, our conformalized predictors consistently achieve nominal coverage with tighter prediction intervals compared to baseline uncertainty estimates. To further account for population heterogeneity, we develop group-conditional conformal bands with formal coverage guarantees across clinically relevant and high-risk subgroups. Finally, we demonstrate the clinical utility of our approach in identifying subjects at risk of progression to Alzheimer’s disease. We introduce an uncertainty-aware progression metric based on the lower conformal bound and show that it enables the identification of 17. 5\% more high-risk subjects compared to standard slope-based methods, highlighting the value of uncertainty calibration in real-world clinical decision making. We make the code available at \href{https: //github. com/vatass/ConformalBiomarkerTrajectories}{\texttt{github. com/vatass/ConformalBiomarkerTrajectories}}.

PDF Details

ICLR Conference 2024 Conference Paper

Adversarial Training Should Be Cast as a Non-Zero-Sum Game

  • Alexander Robey
  • Fabian Latorre
  • George J. Pappas
  • Seyed Hamed Hassani
  • Volkan Cevher

One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data. Despite the promise of this approach, algorithms based on this paradigm have not engendered sufficient levels of robustness and suffer from pathological behavior like robust overfitting. To understand this shortcoming, we first show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on the robustness of trained classifiers. The identification of this pitfall informs a novel non-zero-sum bilevel formulation of adversarial training, wherein each player optimizes a different objective function. Our formulation yields a simple algorithmic framework that matches and in some cases outperforms state-of-the-art attacks, attains comparable levels of robustness to standard adversarial training algorithms, and does not suffer from robust overfitting.

Details

AAAI Conference 2024 Conference Paper

Conformal Prediction Regions for Time Series Using Linear Complementarity Programming

  • Matthew Cleaveland
  • Insup Lee
  • George J. Pappas
  • Lars Lindemann

Conformal prediction is a statistical tool for producing prediction regions of machine learning models that are valid with high probability. However, applying conformal prediction to time series data leads to conservative prediction regions. In fact, to obtain prediction regions over T time steps with confidence 1--delta, previous works require that each individual prediction region is valid with confidence 1--delta/T. We propose an optimization-based method for reducing this conservatism to enable long horizon planning and verification when using learning-enabled time series predictors. Instead of considering prediction errors individually at each time step, we consider a parameterized prediction error over multiple time steps. By optimizing the parameters over an additional dataset, we find prediction regions that are not conservative. We show that this problem can be cast as a mixed integer linear complementarity program (MILCP), which we then relax into a linear complementarity program (LCP). Additionally, we prove that the relaxed LP has the same optimal cost as the original MILCP. Finally, we demonstrate the efficacy of our method on case studies using pedestrian trajectory predictors and F16 fighter jet altitude predictors.

PDF Details DOI

ICML Conference 2024 Conference Paper

Conformal Prediction with Learned Features

  • Shayan Kiyani
  • George J. Pappas
  • Seyed Hamed Hassani

In this paper, we focus on the problem of conformal prediction with conditional guarantees. Prior work has shown that it is impossible to construct nontrivial prediction sets with full conditional coverage guarantees. A wealth of research has considered relaxations of full conditional guarantees, relying on some predefined uncertainty structures. Departing from this line of thinking, we propose Partition Learning Conformal Prediction (PLCP), a framework to improve conditional validity of prediction sets through learning uncertainty-guided features from the calibration data. We implement PLCP efficiently with alternating gradient descent, utilizing off-the-shelf machine learning models. We further analyze PLCP theoretically and provide conditional guarantees for infinite and finite sample sizes. Finally, our experimental results over four real-world and synthetic datasets show the superior performance of PLCP compared to state-of-the-art methods in terms of coverage and length in both classification and regression scenarios.

Details

TMLR Journal 2024 Journal Article

Federated TD Learning with Linear Function Approximation under Environmental Heterogeneity

  • Han Wang
  • Aritra Mitra
  • Hamed Hassani
  • George J. Pappas
  • James Anderson

We initiate the study of federated reinforcement learning under environmental heterogeneity by considering a policy evaluation problem. Our setup involves $N$ agents interacting with environments that share the same state and action space but differ in their reward functions and state transition kernels. Assuming agents can communicate via a central server, we ask: \textit{Does exchanging information expedite the process of evaluating a common policy?} To answer this question, we provide the first comprehensive finite-time analysis of a federated temporal difference (TD) learning algorithm with linear function approximation, while accounting for Markovian sampling, heterogeneity in the agents' environments, and multiple local updates to save communication. Our analysis crucially relies on several novel ingredients: (i) deriving perturbation bounds on TD fixed points as a function of the heterogeneity in the agents' underlying Markov decision processes (MDPs); (ii) introducing a virtual MDP to closely approximate the dynamics of the federated TD algorithm; and (iii) using the virtual MDP to make explicit connections to federated optimization. Putting these pieces together, we prove that in a low-heterogeneity regime, exchanging model estimates leads to linear convergence speedups in the number of agents. Our theoretical contribution is significant in that it is the first result of its kind in multi-agent/federated reinforcement learning that complements the numerous analogous results in heterogeneous federated optimization.

PDF Details

ICML Conference 2024 Conference Paper

Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples

  • Thomas T. C. K. Zhang
  • Bruce D. Lee
  • Ingvar M. Ziemann
  • George J. Pappas
  • Nikolai Matni

A driving force behind the diverse applicability of modern machine learning is the ability to extract meaningful features across many sources. However, many practical domains involve data that are non-identically distributed across sources, and possibly statistically dependent within its source, violating vital assumptions in existing theoretical studies of representation learning. Toward addressing these issues, we establish statistical guarantees for learning general nonlinear representations from multiple data sources that admit different input distributions and possibly dependent data. Specifically, we study the sample-complexity of learning $T+1$ functions $f_\star^{(t)} \circ g_\star$ from a function class $\mathcal{F} \times \mathcal{G}$, where $f_\star^{(t)}$ are task specific linear functions and $g_\star$ is a shared non-linear representation. An approximate representation $\hat g$ is estimated using $N$ samples from each of $T$ source tasks, and a fine-tuning function $\hat f^{(0)}$ is fit using $N’$ samples from a target task passed through $\hat g$. Our results show that the excess risk of the estimate $\hat f^{(0)} \circ \hat g$ on the target task decays as $\tilde{\mathcal{O}}\Big(\frac{\mathrm{C}(\mathcal{G})}{N T} + \frac{\text{dim}(\mathcal{F})}{N’}\Big)$, where $\mathrm{C}(\mathcal{G})$ denotes the complexity of $\mathcal{G}$. Notably, our rates match that of the iid setting, while requiring fewer samples per task than prior analysis and admitting no dependence on the mixing time. We support our analysis with numerical experiments performing imitation learning over non-linear dynamical systems.

Details

NeurIPS Conference 2024 Conference Paper

JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models

  • Patrick Chao
  • Edoardo Debenedetti
  • Alexander Robey
  • Maksym Andriushchenko
  • Francesco Croce
  • Vikash Sehwag
  • Edgar Dobriban
  • Nicolas Flammarion

Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. Evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. First, there is no clear standard of practice regarding jailbreaking evaluation. Second, existing works compute costs and success rates in incomparable ways. And third, numerous works are not reproducible, as they withhold adversarial prompts, involve closed-source code, or rely on evolving proprietary APIs. To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors---both original and sourced from prior work---which align with OpenAI's usage policies; (3) a standardized evaluation framework at https: //github. com/JailbreakBench/jailbreakbench that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard at https: //jailbreakbench. github. io/ that tracks the performance of attacks and defenses for various LLMs. We have carefully considered the potential ethical implications of releasing this benchmark, and believe that it will be a net positive for the community.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Optimal Scene Graph Planning with Large Language Model Guidance

  • Zhirui Dai
  • Arash Asgharivaskasi
  • Thai Duong 0001
  • Shusen Lin
  • Maria-Elizabeth Tzes
  • George J. Pappas
  • Nikolay Atanasov 0001

Recent advances in metric, semantic, and topological mapping have equipped autonomous robots with concept grounding capabilities to interpret natural language tasks. Leveraging these capabilities, this work develops an efficient task planning algorithm for hierarchical metric-semantic models. We consider a scene graph model of the environment and utilize a large language model (LLM) to convert a natural language task into a linear temporal logic (LTL) automaton. Our main contribution is to enable optimal hierarchical LTL planning with LLM guidance over scene graphs. To achieve efficiency, we construct a hierarchical planning domain that captures the attributes and connectivity of the scene graph and the task automaton, and provide semantic guidance via an LLM heuristic function. To guarantee optimality, we design an LTL heuristic function that is provably consistent and supplements the potentially inadmissible LLM guidance in multi-heuristic planning. We demonstrate efficient planning of complex natural language tasks in scene graphs of virtualized real environments.

Details

ICML Conference 2024 Conference Paper

Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Loss

  • Ingvar M. Ziemann
  • Stephen Tu
  • George J. Pappas
  • Nikolai Matni

In this work, we study statistical learning with dependent data and square loss in a hypothesis class with tail decay in Orlicz space: $\mathscr{F}\subset L_{\Psi_p}$. Our inquiry is motivated by the search for a sharp noise interaction term, or variance proxy, in learning with dependent (e. g. $\beta$-mixing) data. Typical non-asymptotic results exhibit variance proxies that are deflated multiplicatively in the mixing time of the underlying covariates process. We show that whenever the topologies of $L^2$ and $\Psi_p$ are comparable on our hypothesis class $\mathscr{F}$, the empirical risk minimizer achieves a rate that only depends on the complexity of the class and second order statistics in its leading term. We refer to this as a near mixing-free rate, since direct dependence on mixing is relegated to an additive higher order term. Our approach, reliant on mixed tail generic chaining, allows us to obtain sharp, instance-optimal rates. Examples that satisfy our framework include for instance sub-Gaussian linear regression and bounded smoothness classes.

Details

TMLR Journal 2024 Journal Article

Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning

  • Aritra Mitra
  • George J. Pappas
  • Hamed Hassani

In large-scale distributed machine learning, recent works have studied the effects of compressing gradients in stochastic optimization to alleviate the communication bottleneck. These works have collectively revealed that stochastic gradient descent (SGD) is robust to structured perturbations such as quantization, sparsification, and delays. Perhaps surprisingly, despite the surge of interest in multi-agent reinforcement learning, almost nothing is known about the analogous question: \textit{Are common reinforcement learning (RL) algorithms also robust to similar perturbations?} We investigate this question by studying a variant of the classical temporal difference (TD) learning algorithm with a perturbed update direction, where a general compression operator is used to model the perturbation. Our work makes three important technical contributions. First, we prove that compressed TD algorithms, coupled with an error-feedback mechanism used widely in optimization, exhibit the same non-asymptotic theoretical guarantees as their SGD counterparts. Second, we show that our analysis framework extends seamlessly to nonlinear stochastic approximation schemes that subsume Q-learning. Third, we prove that for multi-agent TD learning, one can achieve linear convergence speedups with respect to the number of agents while communicating just $\tilde{O}(1)$ bits per iteration. Notably, these are the first finite-time results in RL that account for general compression operators and error-feedback in tandem with linear function approximation and Markovian sampling. Our proofs hinge on the construction of novel Lyapunov functions that capture the dynamics of a memory variable introduced by error-feedback.

PDF Details

IROS Conference 2023 Conference Paper

Enhancing Sample Efficiency and Uncertainty Compensation in Learning-Based Model Predictive Control for Aerial Robots

  • Kong Yao Chee
  • Thales C. Silva
  • M. Ani Hsieh
  • George J. Pappas

The recent increase in data availability and reliability has led to a surge in the development of learning-based model predictive control (MPC) frameworks for robot systems. Despite attaining substantial performance improvements over their non-learning counterparts, many of these frameworks rely on an offline learning procedure to synthesize a dynamics model. This implies that uncertainties encountered by the robot during deployment are not accounted for in the learning process. On the other hand, learning-based MPC methods that learn dynamics models online are computationally expensive and often require a significant amount of data. To alleviate these shortcomings, we propose a novel learning-enhanced MPC framework that incorporates components from C 1 adaptive control into learning-based MPC. This integration enables the accurate compensation of both matched and unmatched uncertainties in a sample-efficient way, enhancing the control performance during deployment. In our proposed framework, we present two variants and apply them to the control of a quadrotor system. Through simulations and physical experiments, we demonstrate that the proposed framework not only allows the synthesis of an accurate dynamics model on-the-fly, but also significantly improves the closed-loop control performance under a wide range of spatio-temporal uncertainties.

Details

ICRA Conference 2023 Conference Paper

Graph Neural Networks for Multi-Robot Active Information Acquisition

  • Mariliza Tzes
  • Nikolaos Bousias
  • Evangelos Chatzipantazis
  • George J. Pappas

This paper addresses the Multi-Robot Active In-formation Acquisition (AIA) problem, where a team of mobile robots, communicating through an underlying graph, estimates a hidden state expressing a phenomenon of interest. Applications like target tracking, coverage and SLAM can be expressed in this framework. Existing approaches, though, are either not scalable, unable to handle dynamic phenomena or not robust to changes in the communication graph. To counter these shortcomings, we propose an Information-aware Graph Block Network (I-GBNet), an AIA adaptation of Graph Neural Networks, that aggregates information over the graph represen-tation and provides sequential-decision making in a distributed manner. The I-GBNet, trained via imitation learning with a centralized sampling-based expert solver, exhibits permutation equivariance and time invariance, while harnessing the superior scalability, robustness and generalizability to previously unseen environments and robot configurations. Numerical simulations on significantly larger graphs and dimensionality of the hidden state and more complex environments than those seen in training validate the properties of the proposed architecture and its efficacy in the application of localization and tracking of dynamic targets.

Details

ICRA Conference 2023 Conference Paper

Multi-Robot Mission Planning in Dynamic Semantic Environments

  • Samarth Kalluraya
  • George J. Pappas
  • Yiannis Kantaros

This paper addresses a new semantic multi-robot planning problem in uncertain and dynamic environments. Particularly, the environment is occupied with mobile and uncertain semantic targets. These targets are governed by stochastic dynamics while their current and future positions as well as their semantic labels are uncertain. Our goal is to control mobile sensing robots so that they can accomplish collaborative semantic tasks defined over the uncertain current/future positions and semantic labels of these targets. We express these tasks using Linear Temporal Logic (LTL). We propose a sampling-based approach that explores the robot motion space, the mission specification space, as well as the future configurations of the semantic targets to design optimal paths. These paths are revised online to adapt to uncertain perceptual feedback. To the best of our knowledge, this is the first work that addresses semantic mission planning problems in uncertain and dynamic semantic environments. We provide extensive experiments that demonstrate the efficiency of the proposed method.

Details

IROS Conference 2023 Conference Paper

Robust Localization of Aerial Vehicles via Active Control of Identical Ground Vehicles

  • Igor Spasojevic
  • Xu Liu 0007
  • Ankit Prabhu
  • Alejandro Ribeiro
  • George J. Pappas
  • Vijay Kumar 0001

This paper addresses the problem of active collaborative localization in heterogeneous robot teams with unknown data association. It involves positioning a small number of identical unmanned ground vehicles (UGVs) at desired positions so that an unmanned aerial vehicle (UAV) can, through unlabelled measurements of UGVs, uniquely determine its global pose. We model the problem as a sequential two player game, in which the first player positions the UGVs and the second identifies the two distinct hypothetical poses of the UAV at which the sets of measurements to the UGVs differ by as little as possible. We solve the underlying problem from the vantage point of the first player for a subclass of measurement models using a mixture of local optimization and exhaustive search procedures. Real-world experiments with a team of UAV and UGVs show that our method can achieve centimeter-level global localization accuracy. We also show that our method consistently outperforms random positioning of UGVs by a large margin, with as much as a 90% reduction in position and angular estimation error. Our method can tolerate a significant amount of random as well as non-stochastic measurement noise. This indicates its potential for reliable state estimation on board size, weight, and power (SWaP) constrained UAVs. This work enables robust localization in perceptually-challenged GPS-denied environments, thus paving the road for large-scale multi-robot navigation and mapping.

Details

ICRA Conference 2023 Conference Paper

Socially Fair Coverage Control

  • Matthew Malencia
  • George J. Pappas
  • Vijay Kumar 0001

We investigate and develop algorithms for social fairness in coverage control problems. Existing coverage control methods are efficient, optimizing the average expected distance from any event to the nearest robot. However, in societal applications like disaster response or transportation, these conventional objectives lead to disparate coverage costs with respect to different groups within a population. We formulate social fairness for coverage control as the minimization of the maximum coverage cost among a set of groups within a population. Our approach uses Voronoi iteration to solve this novel problem by approximating the non-differentiable objective with the log-sum-exp and defining a gradient based controller that prioritizes fairness while also optimizing average performance when disparities between groups are low. We show convergence properties of this proposed control law and demonstrate the approach in simulations of randomly generated population densities as well as environments generated from U. S. census data on population rates and demographics. Our approach provides greater fairness than existing methods while maintaining similar computational time and convergence properties.

Details

NeurIPS Conference 2023 Conference Paper

The noise level in linear regression with dependent data

  • Ingvar Ziemann
  • Stephen Tu
  • George J. Pappas
  • Nikolai Matni

We derive upper bounds for random design linear regression with dependent ($\beta$-mixing) data absent any realizability assumptions. In contrast to the strictly realizable martingale noise regime, no sharp \emph{instance-optimal} non-asymptotics are available in the literature. Up to constant factors, our analysis correctly recovers the variance term predicted by the Central Limit Theorem---the noise level of the problem---and thus exhibits graceful degradation as we introduce misspecification. Past a burn-in, our result is sharp in the moderate deviations regime, and in particular does not inflate the leading order term by mixing time factors.

PDF Details

ICML Conference 2023 Conference Paper

Variational Autoencoding Neural Operators

  • Jacob H. Seidman
  • Georgios Kissas
  • George J. Pappas
  • Paris Perdikaris

Unsupervised learning with functional data is an emerging paradigm of machine learning research with applications to computer vision, climate modeling and physical systems. A natural way of modeling functional data is by learning operators between infinite dimensional spaces, leading to discretization invariant representations that scale independently of the sample grid resolution. Here we present Variational Autoencoding Neural Operators (VANO), a general strategy for making a large class of operator learning architectures act as variational autoencoders. For this purpose, we provide a novel rigorous mathematical formulation of the variational objective in function spaces for training. VANO first maps an input function to a distribution over a latent space using a parametric encoder and then decodes a sample from the latent distribution to reconstruct the input, as in classic variational autoencoders. We test VANO with different model set-ups and architecture choices for a variety of benchmarks. We start from a simple Gaussian random field where we can analytically track what the model learns and progressively transition to more challenging benchmarks including modeling phase separation in Cahn-Hilliard systems and real world satellite data for measuring Earth surface deformation.

Details

IROS Conference 2022 Conference Paper

Adaptive Sampling of Latent Phenomena using Heterogeneous Robot Teams (ASLaP-HR)

  • Matthew Malencia
  • Sandeep Manjanna
  • M. Ani Hsieh
  • George J. Pappas
  • Vijay Kumar 0001

In this paper, we present an online adaptive planning strategy for a team of robots with heterogeneous sensors to sample from a latent spatial field using a learned model for decision making. Current robotic sampling methods seek to gather information about an observable spatial field. However, many applications, such as environmental monitoring and precision agriculture, involve phenomena that are not directly observable or are costly to measure, called latent phenomena. In our approach, we seek to reason about the latent phenomenon in real-time by effectively sampling the observable spatial fields using a team of robots with heterogeneous sensors, where each robot has a distinct sensor to measure a different observable field. The information gain is estimated using a learned model that maps from the observable spatial fields to the latent phenomenon. This model captures aleatoric uncertainty in the relationship to allow for information theoretic measures. Additionally, we explicitly consider the correlations among the observable spatial fields, capturing the relationship between sensor types whose observations are not independent. We show it is possible to learn these correlations, and investigate the impact of the learned correlation models on the performance of our sampling approach. Through our qualitative and quantitative results, we illustrate that empirically learned correlations improve the overall sampling efficiency of the team. We simulate our approach using a data set of sensor measurements collected on Lac Hertel, in Quebec, which we make publicly available.

Details

NeurIPS Conference 2022 Conference Paper

Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds

  • Aritra Mitra
  • Arman Adibi
  • George J. Pappas
  • Hamed Hassani

We consider a linear stochastic bandit problem involving $M$ agents that can collaborate via a central server to minimize regret. A fraction $\alpha$ of these agents are adversarial and can act arbitrarily, leading to the following tension: while collaboration can potentially reduce regret, it can also disrupt the process of learning due to adversaries. In this work, we provide a fundamental understanding of this tension by designing new algorithms that balance the exploration-exploitation trade-off via carefully constructed robust confidence intervals. We also complement our algorithms with tight analyses. First, we develop a robust collaborative phased elimination algorithm that achieves $\tilde{O}\left(\alpha+ 1/\sqrt{M}\right) \sqrt{dT}$ regret for each good agent; here, $d$ is the model-dimension and $T$ is the horizon. For small $\alpha$, our result thus reveals a clear benefit of collaboration despite adversaries. Using an information-theoretic argument, we then prove a matching lower bound, thereby providing the first set of tight, near-optimal regret bounds for collaborative linear bandits with adversaries. Furthermore, by leveraging recent advances in high-dimensional robust statistics, we significantly extend our algorithmic ideas and results to (i) the generalized linear bandit model that allows for non-linear observation maps; and (ii) the contextual bandit setting that allows for time-varying feature vectors.

PDF Details

ICLR Conference 2022 Conference Paper

Do deep networks transfer invariances across classes?

  • Allan Zhou
  • Fahim Tajwar
  • Alexander Robey
  • Tom Knowles
  • George J. Pappas
  • Seyed Hamed Hassani
  • Chelsea Finn

In order to generalize well, classifiers must learn to be invariant to nuisance transformations that do not alter an input's class. Many problems have "class-agnostic" nuisance transformations that apply similarly to all classes, such as lighting and background changes for image classification. Neural networks can learn these invariances given sufficient data, but many real-world datasets are heavily class imbalanced and contain only a few examples for most of the classes. We therefore pose the question: how well do neural networks transfer class-agnostic invariances learned from the large classes to the small ones? Through careful experimentation, we observe that invariance to class-agnostic transformations is still heavily dependent on class size, with the networks being much less invariant on smaller classes. This result holds even when using data balancing techniques, and suggests poor invariance transfer across classes. Our results provide one explanation for why classifiers generalize poorly on unbalanced and long-tailed distributions. Based on this analysis, we show how a generative approach for learning the nuisance transformations can help transfer invariances across classes and improve performance on a set of imbalanced image classification benchmarks.

Details

JMLR Journal 2022 Journal Article

Learning Operators with Coupled Attention

  • Georgios Kissas
  • Jacob H. Seidman
  • Leonardo Ferreira Guilhoto
  • Victor M. Preciado
  • George J. Pappas
  • Paris Perdikaris

Supervised operator learning is an emerging machine learning paradigm with applications to modeling the evolution of spatio-temporal dynamical systems and approximating general black-box relationships between functional data. We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. In our architecture, the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions, enabling us to approximate nonlinear operators even when the number of output function measurements in the training set is very small. Our formulation is accompanied by rigorous approximation theoretic guarantees on the universal expressiveness of the proposed model. Empirically, we evaluate the performance of LOCA on several operator learning scenarios involving systems governed by ordinary and partial differential equations, as well as a black-box climate prediction problem. Through these scenarios we demonstrate state of the art accuracy, robustness with respect to noisy input data, and a consistently small spread of errors over testing data sets, even for out-of-distribution prediction tasks. [abs] [ pdf ][ bib ] &copy JMLR 2022. ( edit, beta )

PDF Details

NeurIPS Conference 2022 Conference Paper

NOMAD: Nonlinear Manifold Decoders for Operator Learning

  • Jacob Seidman
  • Georgios Kissas
  • Paris Perdikaris
  • George J. Pappas

Supervised learning in function spaces is an emerging area of machine learning research with applications to the prediction of complex physical systems such as fluid flows, solid mechanics, and climate modeling. By directly learning maps (operators) between infinite dimensional function spaces, these models are able to learn discretization invariant representations of target functions. A common approach is to represent such target functions as linear combinations of basis elements learned from data. However, there are simple scenarios where, even though the target functions form a low dimensional submanifold, a very large number of basis elements is needed for an accurate linear representation. Here we present NOMAD, a novel operator learning framework with a nonlinear decoder map capable of learning finite dimensional representations of nonlinear submanifolds in function spaces. We show this method is able to accurately learn low dimensional representations of solution manifolds to partial differential equations while outperforming linear models of larger size. Additionally, we compare to state-of-the-art operator learning methods on a complex fluid dynamics benchmark and achieve competitive performance with a significantly smaller model size and training cost.

PDF Details

ICML Conference 2022 Conference Paper

Probabilistically Robust Learning: Balancing Average and Worst-case Performance

  • Alexander Robey
  • Luiz F. O. Chamon
  • George J. Pappas
  • Seyed Hamed Hassani

Many of the successes of machine learning are based on minimizing an averaged loss function. However, it is well-known that this paradigm suffers from robustness issues that hinder its applicability in safety-critical domains. These issues are often addressed by training against worst-case perturbations of data, a technique known as adversarial training. Although empirically effective, adversarial training can be overly conservative, leading to unfavorable trade-offs between nominal performance and robustness. To this end, in this paper we propose a framework called probabilistic robustness that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case by enforcing robustness to most rather than to all perturbations. From a theoretical point of view, this framework overcomes the trade-offs between the performance and the sample-complexity of worst-case and average-case learning. From a practical point of view, we propose a novel algorithm based on risk-aware optimization that effectively balances average- and worst-case performance at a considerably lower computational cost relative to adversarial training. Our results on MNIST, CIFAR-10, and SVHN illustrate the advantages of this framework on the spectrum from average- to worst-case robustness. Our code is available at: https: //github. com/arobey1/advbench.

Details

NeurIPS Conference 2022 Conference Paper

Probable Domain Generalization via Quantile Risk Minimization

  • Cian Eastwood
  • Alexander Robey
  • Shashank Singh
  • Julius von Kügelgen
  • Hamed Hassani
  • George J. Pappas
  • Bernhard Schölkopf

Domain generalization (DG) seeks predictors which perform well on unseen test distributions by leveraging data drawn from multiple related training distributions or domains. To achieve this, DG is commonly formulated as an average- or worst-case problem over the set of possible domains. However, predictors that perform well on average lack robustness while predictors that perform well in the worst case tend to be overly-conservative. To address this, we propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability. Our key idea is that distribution shifts seen during training should inform us of probable shifts at test time, which we realize by explicitly relating training and test domains as draws from the same underlying meta-distribution. To achieve probable DG, we propose a new optimization problem called Quantile Risk Minimization (QRM). By minimizing the $\alpha$-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability $\alpha$. To solve QRM in practice, we propose the Empirical QRM (EQRM) algorithm and provide: (i) a generalization bound for EQRM; and (ii) the conditions under which EQRM recovers the causal predictor as $\alpha \to 1$. In our experiments, we introduce a more holistic quantile-focused evaluation protocol for DG, and demonstrate that EQRM outperforms state-of-the-art baselines on datasets from WILDS and DomainBed.

PDF Details

ICRA Conference 2022 Conference Paper

Reactive Informative Planning for Mobile Manipulation Tasks under Sensing and Environmental Uncertainty

  • Mariliza Tzes
  • Vasileios Vasilopoulos
  • Yiannis Kantaros
  • George J. Pappas

In this paper we address mobile manipulation planning problems in the presence of sensing and environmental uncertainty. In particular, we consider mobile sensing manipulators operating in environments with unknown geometry and uncertain movable objects, while being responsible for accomplishing tasks requiring grasping and releasing objects in a logical fashion. Existing algorithms either do not scale well or neglect sensing and/or environmental uncertainty. To face these challenges, we propose a hybrid control architecture, where a symbolic controller generates high-level manipulation commands (e. g. , grasp an object) based on environmental feedback, an informative planner designs paths to actively decrease the uncertainty of objects of interest, and a continuous reactive controller tracks the sparse waypoints comprising the informative paths while avoiding a priori unknown obstacles. The overall architecture can handle environmental and sensing uncertainty online, as the robot explores its workspace. Using numerical simulations, we show that the proposed architecture can handle tasks of increased complexity while responding to unanticipated adverse configurations.

Details

AIJ Journal 2022 Journal Article

Risk verification of stochastic systems with neural network controllers

  • Matthew Cleaveland
  • Lars Lindemann
  • Radoslav Ivanov
  • George J. Pappas

Motivated by the fragility of neural network (NN) controllers in safety-critical applications, we present a data-driven framework for verifying the risk of stochastic dynamical systems with NN controllers. Given a stochastic control system, an NN controller, and a specification equipped with a notion of trace robustness (e. g. , constraint functions or signal temporal logic), we collect trajectories from the system that may or may not satisfy the specification. In particular, each of the trajectories produces a robustness value that indicates how well (severely) the specification is satisfied (violated). We then compute risk metrics over these robustness values to estimate the risk that the NN controller will not satisfy the specification. We are further interested in quantifying the difference in risk between two systems, and we show how the risk estimated from a nominal system can provide an upper bound the risk of a perturbed version of the system. In particular, the tightness of this bound depends on the closeness of the systems in terms of the closeness of their system trajectories. For Lipschitz continuous and incrementally input-to-state stable systems, we show how to exactly quantify system closeness with varying degrees of conservatism, while we estimate system closeness for more general systems from data in our experiments. We demonstrate our risk verification approach on two case studies, an underwater vehicle and an F1/10 autonomous car.

Details DOI

NeurIPS Conference 2021 Conference Paper

Adversarial Robustness with Semi-Infinite Constrained Learning

  • Alexander Robey
  • Luiz Chamon
  • George J. Pappas
  • Hamed Hassani
  • Alejandro Ribeiro

Despite strong performance in numerous applications, the fragility of deep learning to input perturbations has raised serious questions about its use in safety-critical domains. While adversarial training can mitigate this issue in practice, state-of-the-art methods are increasingly application-dependent, heuristic in nature, and suffer from fundamental trade-offs between nominal performance and robustness. Moreover, the problem of finding worst-case perturbations is non-convex and underparameterized, both of which engender a non-favorable optimization landscape. Thus, there is a gap between the theory and practice of robust learning, particularly with respect to when and why adversarial training works. In this paper, we take a constrained learning approach to address these questions and to provide a theoretical foundation for robust learning. In particular, we leverage semi-infinite optimization and non-convex duality theory to show that adversarial training is equivalent to a statistical problem over perturbation distributions. Notably, we show that a myriad of previous robust training techniques can be recovered for particular, sub-optimal choices of these distributions. Using these insights, we then propose a hybrid Langevin Markov Chain Monte Carlo approach for which several common algorithms (e. g. , PGD) are special cases. Finally, we show that our approach can mitigate the trade-off between nominal and robust performance, yielding state-of-the-art results on MNIST and CIFAR-10. Our code is available at: https: //github. com/arobey1/advbench.

PDF Details

ICRA Conference 2021 Conference Paper

Deep Reinforcement Learning for Active Target Tracking

  • Heejin Jeong
  • Seyed Hamed Hassani
  • Manfred Morari
  • Daniel D. Lee
  • George J. Pappas

We solve active target tracking, one of the essential tasks in autonomous systems, using a deep reinforcement learning (RL) approach. In this problem, an autonomous agent is tasked with acquiring information about targets of interests using its on-board sensors. The classical challenges in this problem are system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. RL provides solutions for these challenges as the length of its effective planning horizon does not affect the computational complexity, and it drops the strong dependency of an algorithm on system models. In particular, we introduce Active Tracking Target Network (ATTN), a unified deep RL policy that is capable of solving major sub-tasks of active target tracking – in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model. Additionally, the same policy is able to navigate in obstacle environments to reach distant targets as well as explore the environment when targets are positioned in unexpected locations.

Details

IROS Conference 2021 Conference Paper

Distributed Sampling-based Planning for Non-Myopic Active Information Gathering

  • Mariliza Tzes
  • Yiannis Kantaros
  • George J. Pappas

This paper addresses the problem of active information gathering for multi-robot systems. Specifically, we consider scenarios where robots are tasked with reducing uncertainty of dynamical hidden states evolving in complex environments. The majority of existing information gathering approaches are centralized and, therefore, they cannot be applied to distributed robot teams where communication to a central user is not available. To address this challenge, we propose a novel distributed sampling-based planning algorithm that can significantly increase robot and target scalability while decreasing computational cost. In our non-myopic approach, all robots build in parallel local trees exploring the information space and their corresponding motion space. As the robots construct their respective local trees, they communicate with their neighbors to exchange and aggregate their local beliefs about the hidden state through a distributed Kalman filter. We show that the proposed algorithm is probabilistically complete and asymptotically optimal. We provide extensive simulation results that demonstrate the scalability of the proposed algorithm and that it can address large-scale, multi-robot information gathering tasks, that are computationally challenging for centralized methods.

Details

NeurIPS Conference 2021 Conference Paper

Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients

  • Aritra Mitra
  • Rayana Jaafar
  • George J. Pappas
  • Hamed Hassani

We consider a standard federated learning (FL) setup where a group of clients periodically coordinate with a central server to train a statistical model. We develop a general algorithmic framework called FedLin to tackle some of the key challenges intrinsic to FL, namely objective heterogeneity, systems heterogeneity, and infrequent and imprecise communication. Our framework is motivated by the observation that under these challenges, various existing FL algorithms suffer from a fundamental speed-accuracy conflict: they either guarantee linear convergence but to an incorrect point, or convergence to the global minimum but at a sub-linear rate, i. e. , fast convergence comes at the expense of accuracy. In contrast, when the clients' local loss functions are smooth and strongly convex, we show that FedLin guarantees linear convergence to the global minimum, despite arbitrary objective and systems heterogeneity. We then establish matching upper and lower bounds on the convergence rate of FedLin that highlight the effects of infrequent, periodic communication. Finally, we show that FedLin preserves linear convergence rates under aggressive gradient sparsification, and quantify the effect of the compression level on the convergence rate. Notably, our work is the first to provide tight linear convergence rate guarantees, and constitutes the first comprehensive analysis of gradient sparsification in FL.

PDF Details

NeurIPS Conference 2021 Conference Paper

Model-Based Domain Generalization

  • Alexander Robey
  • George J. Pappas
  • Hamed Hassani

Despite remarkable success in a variety of applications, it is well-known that deep learning can fail catastrophically when presented with out-of-distribution data. Toward addressing this challenge, we consider the \emph{domain generalization} problem, wherein predictors are trained using data drawn from a family of related training domains and then evaluated on a distinct and unseen test domain. We show that under a natural model of data generation and a concomitant invariance condition, the domain generalization problem is equivalent to an infinite-dimensional constrained statistical learning problem; this problem forms the basis of our approach, which we call Model-Based Domain Generalization. Due to the inherent challenges in solving constrained optimization problems in deep learning, we exploit nonconvex duality theory to develop unconstrained relaxations of this statistical problem with tight bounds on the duality gap. Based on this theoretical motivation, we propose a novel domain generalization algorithm with convergence guarantees. In our experiments, we report improvements of up to 30% over state-of-the-art domain generalization baselines on several benchmarks including ColoredMNIST, Camelyon17-WILDS, FMoW-WILDS, and PACS.

PDF Details

ICRA Conference 2021 Conference Paper

Non-Monotone Energy-Aware Information Gathering for Heterogeneous Robot Teams

  • Xiaoyi Cai
  • Brent Schlotfeldt
  • Kasra Khosoussi
  • Nikolay Atanasov 0001
  • George J. Pappas
  • Jonathan P. How

This paper considers the problem of planning trajectories for a team of sensor-equipped robots to reduce uncertainty about a dynamical process. Optimizing the trade-off between information gain and energy cost (e. g. , control effort, distance travelled) is desirable but leads to a non-monotone objective function in the set of robot trajectories. Therefore, common multi-robot planning algorithms based on techniques such as coordinate descent lose their performance guarantees. Methods based on local search provide performance guarantees for optimizing a non-monotone submodular function, but require access to all robots’ trajectories, making it not suitable for distributed execution. This work proposes a distributed planning approach based on local search and shows how lazy/greedy methods can be adopted to reduce the computation and communication of the approach. We demonstrate the efficacy of the proposed method by coordinating robot teams composed of both ground and aerial vehicles with different sensing/control profiles and evaluate the algorithm’s performance in two target tracking scenarios. Compared to the naive distributed execution of local search, our approach saves up to 60% communication and 80–92% computation on average when coordinating up to 10 robots, while outperforming the coordinate descent based algorithm in achieving a desirable trade-off between sensing and energy cost.

Details

ICRA Conference 2021 Conference Paper

Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments

  • Vasileios Vasilopoulos
  • Yiannis Kantaros
  • George J. Pappas
  • Daniel E. Koditschek

Complex manipulation tasks, such as rearrangement planning of numerous objects, are combinatorially hard problems. Existing algorithms either do not scale well or assume a great deal of prior knowledge about the environment, and few offer any rigorous guarantees. In this paper, we propose a novel hybrid control architecture for achieving such tasks with mobile manipulators. On the discrete side, we enrich a temporal logic specification with mobile manipulation primitives such as moving to a point, and grasping or moving an object. Such specifications are translated to an automaton representation, which orchestrates the physical grounding of the task to mobility or manipulation controllers. The grounding from the discrete to the continuous reactive controller is online and can respond to the discovery of unknown obstacles or decide to push out of the way movable objects that prohibit task accomplishment. Despite the problem complexity, we prove that, under specific conditions, our architecture enjoys provable completeness on the discrete side, provable termination on the continuous side, and avoids all obstacles in the environment. Simulations illustrate the efficiency of our architecture that can handle tasks of increased complexity while also responding to unknown obstacles or unanticipated adverse configurations.

Details

NeurIPS Conference 2021 Conference Paper

Safe Pontryagin Differentiable Programming

  • Wanxin Jin
  • Shaoshuai Mou
  • George J. Pappas

We propose a Safe Pontryagin Differentiable Programming (Safe PDP) methodology, which establishes a theoretical and algorithmic framework to solve a broad class of safety-critical learning and control tasks---problems that require the guarantee of safety constraint satisfaction at any stage of the learning and control progress. In the spirit of interior-point methods, Safe PDP handles different types of system constraints on states and inputs by incorporating them into the cost or loss through barrier functions. We prove three fundamentals of the proposed Safe PDP: first, both the solution and its gradient in the backward pass can be approximated by solving their more efficient unconstrained counterparts; second, the approximation for both the solution and its gradient can be controlled for arbitrary accuracy by a barrier parameter; and third, importantly, all intermediate results throughout the approximation and optimization strictly respect the constraints, thus guaranteeing safety throughout the entire learning and control process. We demonstrate the capabilities of Safe PDP in solving various safety-critical tasks, including safe policy optimization, safe motion planning, and learning MPCs from demonstrations, on different challenging systems such as 6-DoF maneuvering quadrotor and 6-DoF rocket powered landing.

PDF Details

ICRA Conference 2021 Conference Paper

Scalable Active Information Acquisition for Multi-Robot Systems

  • Yiannis Kantaros
  • George J. Pappas

This paper proposes a novel highly scalable nonmyopic planning algorithm for multi-robot Active Information Acquisition (AIA) tasks. AIA scenarios include target localization and tracking, active SLAM, surveillance, environmental monitoring and others. The objective is to compute control policies for multiple robots which minimize the accumulated uncertainty of a static hidden state over an a priori unknown horizon. The majority of existing AIA approaches are centralized and, therefore, face scaling challenges. To mitigate this issue, we propose an online algorithm that relies on decomposing the AIA task into local tasks via a dynamic space-partitioning method. The local subtasks are formulated online and require the robots to switch between exploration and active information gathering roles depending on their functionality in the environment. The switching process is tightly integrated with optimizing information gathering giving rise to a hybrid control approach. We show that the proposed decomposition-based algorithm is probabilistically complete for homogeneous sensor teams and under linearity and Gaussian assumptions. We provide extensive simulation results showing that the proposed algorithm can address large-scale estimation tasks that are computationally challenging to solve using existing centralized approaches.

Details

IROS Conference 2021 Conference Paper

Scalable Reinforcement Learning Policies for Multi-Agent Control

  • Christopher D. Hsu
  • Heejin Jeong
  • George J. Pappas
  • Pratik Chaudhari

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

Details

IROS Conference 2020 Conference Paper

Adaptive Partitioning for Coordinated Multi-agent Perimeter Defense

  • Douglas G. Macharet
  • Austin K. Chen
  • Daigo Shishika
  • George J. Pappas
  • Vijay Kumar 0001

Multi-Robot Systems have been recently employed in different applications and have advantages over single-robot systems, such as increased robustness and task performance efficiency. We consider such assemblies specifically in the scenario of perimeter defense, where the task is to defend a circular perimeter by intercepting radially approaching targets. Possible intruders appear randomly at a fixed distance from the perimeter and with azimuthal location determined by some unknown probability density. Coordination among multiple defenders is a complex combinatorial optimization problem. In this work, we focus on the following two aspects: (i) estimating the probability density that describes the direction from which the next intruders are going to arrive, and (ii) partitioning of the space so that the defenders focus on capturing a disjoint subset of intruders. Results show that the proposed strategy increases the number of captures over a naive baseline strategy, especially in scenarios with non-uniform spatial distributions of intruder arrival. The proposed approach is also efficient and able to quickly adapt to time-varying intruder distributions.

Details

ICRA Conference 2020 Conference Paper

Distributed Attack-Robust Submodular Maximization for Multi-Robot Planning

  • Lifeng Zhou 0001
  • Vasileios Tzoumas
  • George J. Pappas
  • Pratap Tokekar

We aim to guard swarm-robotics applications against denial-of-service (DoS) attacks that result in withdrawals of robots. We focus on applications requiring the selection of actions for each robot, among a set of available ones, e. g. , which trajectory to follow. Such applications are central in large-scale robotic applications, e. g. , multi-robot motion planning for target tracking. But the current attack-robust algorithms are centralized, and scale quadratically with the problem size (e. g. , number of robots). In this paper, we propose a general-purpose distributed algorithm towards robust optimization at scale, with local communications only. We name it distributed robust maximization (DRM). DRM proposes a divide-and-conquer approach that distributively partitions the problem among K cliques of robots. The cliques optimize in parallel, independently of each other. That way, DRM also offers computational speed-ups up to 1/K 2 the running time of its centralized counterparts. K depends on the robots' communication range, which is given as input to DRM. DRM also achieves a close-to-optimal performance. We demonstrate DRM's performance in Gazebo and MATLAB simulations, in scenarios of active target tracking with multiple robots. We observe DRM achieves significant computational speed-ups (it is 3 to 4 orders faster) and, yet, nearly matches the tracking performance of its centralized counterparts.

Details

ICRA Conference 2020 Conference Paper

Information Theoretic Active Exploration in Signed Distance Fields

  • Kelsey Saulnier
  • Nikolay Atanasov 0001
  • George J. Pappas
  • Vijay Kumar 0001

This paper focuses on exploration and occupancy mapping of unknown environments using a mobile robot. While a truncated signed distance field (TSDF) is a popular, efficient, and highly accurate representation of occupancy, few works have considered optimizing robot sensing trajectories for autonomous TSDF mapping. We propose an efficient approach for maintaining TSDF uncertainty and predicting its evolution from potential future sensor measurements without actually receiving them. Efficient uncertainty prediction is critical for long-horizon optimization of potential sensing trajectories. We develop a deterministic tree-search algorithm that evaluates the information gain between the TSDF distribution and potential observations along sequences of robot motion primitives. Efficient planning is achieved by branch-and-bound pruning of uninformative sensing trajectories. The effectiveness of our active TSDF mapping approach is evaluated in several simulated environments with complex visibility constraints.

Details

ICRA Conference 2020 Conference Paper

Reactive Temporal Logic Planning for Multiple Robots in Unknown Environments

  • Yiannis Kantaros
  • Matthew Malencia
  • Vijay Kumar 0001
  • George J. Pappas

This paper proposes a new reactive mission planning algorithm for multiple robots that operate in unknown environments. The robots are equipped with individual sensors that allow them to collectively learn and continuously update a map of the unknown environment. The goal of the robots is to accomplish complex tasks, captured by global co-safe Linear Temporal Logic (LTL) formulas. The majority of existing temporal logic planning approaches rely on discrete abstractions of the robot dynamics operating in known environments and, as a result, they cannot be applied to the more realistic scenarios where the environment is initially unknown. In this paper, we address this novel challenge by proposing the first reactive, and abstraction-free LTL planning algorithm that can be applied for complex mission planning of multiple robots operating in unknown environments. Our algorithm is reactive in the sense that temporal logic planning is adapting to the updated map of the environment and abstraction-free as it does not rely on designing abstractions of robot dynamics. Our proposed algorithm is complete under mild assumptions on the structure of the environment and the sensor models. Our paper provides extensive numerical simulations and hardware experiments that illustrate the theoretical analysis and show that the proposed algorithm can address complex planning tasks in unknown environments.

Details

IJCAI Conference 2019 Conference Paper

Assumed Density Filtering Q-learning

  • Heejin Jeong
  • Clark Zhang
  • George J. Pappas
  • Daniel D. Lee

While off-policy temporal difference (TD) methods have widely been used in reinforcement learning due to their efficiency and simple implementation, their Bayesian counterparts have not been utilized as frequently. One reason is that the non-linear max operation in the Bellman optimality equation makes it difficult to define conjugate distributions over the value functions. In this paper, we introduce a novel Bayesian approach to off-policy TD methods, called as ADFQ, which updates beliefs on state-action values, Q, through an online Bayesian inference method known as Assumed Density Filtering. We formulate an efficient closed-form solution for the value update by approximately estimating analytic parameters of the posterior of the Q-beliefs. Uncertainty measures in the beliefs not only are used in exploration but also provide a natural regularization for the value update considering all next available actions. ADFQ converges to Q-learning as the uncertainty measures of the Q-beliefs decrease and improves common drawbacks of other Bayesian RL algorithms such as computational complexity. We extend ADFQ with a neural network. Our empirical results demonstrate that ADFQ outperforms comparable algorithms on various Atari 2600 games, with drastic improvements in highly stochastic domains or domains with a large action space.

PDF Details

IROS Conference 2019 Conference Paper

Learning Q-network for Active Information Acquisition

  • Heejin Jeong
  • Brent Schlotfeldt
  • Seyed Hamed Hassani
  • Manfred Morari
  • Daniel D. Lee
  • George J. Pappas

In this paper, we propose a novel Reinforcement Learning approach for solving the Active Information Acquisition problem, which requires an agent to choose a sequence of actions in order to acquire information about a process of interest using on-board sensors. The classic challenges in the information acquisition problem are the dependence of a planning algorithm on known models and the difficulty of computing information-theoretic cost functions over arbitrary distributions. In contrast, the proposed framework of reinforcement learning does not require any knowledge on models and alleviates the problems during an extended training stage. It results in policies that are efficient to execute online and applicable for real-time control of robotic systems. Furthermore, the state-of-the-art planning methods are typically restricted to short horizons, which may become problematic with local minima. Reinforcement learning naturally handles the issue of planning horizon in information problems as it maximizes a discounted sum of rewards over a long finite or infinite time horizon. We discuss the potential benefits of the proposed framework and compare the performance of the novel algorithm to an existing information acquisition method for multi-target tracking scenarios.

Details

IROS Conference 2019 Conference Paper

Maximum Information Bounds for Planning Active Sensing Trajectories

  • Brent Schlotfeldt
  • Nikolay Atanasov 0001
  • George J. Pappas

This paper considers the problem of planning trajectories for robots equipped with sensors whose task is to track an evolving target process in the world. We focus on processes which can be represented by a Gaussian random variable, which is known to reduce the general stochastic information acquisition problem to a deterministic problem, which is much simpler to solve. Previous work on solving the resulting deterministic problem focuses on computing a search tree by Forward Value Iteration and pruning uninformative nodes early on in the search via a domination criteria. In this work we formulate the Active Information Acquisition problem as a deterministic planning problem where algorithms like Dijkstra and $\mathrm{A}^{*}$ can produce optimal solutions. To use $\mathrm{A}^{*}$ effectively in long planning horizons we derive a consistent and admissible heuristic as a function of the sensor model which can be used in information acquisition tasks such as actively mapping static and moving targets in an environment with obstacles. We validate the results in several simulations indicating that the resulting heuristic informed algorithm can recover optimal solutions faster than existing search-based methods.

Details

IROS Conference 2019 Conference Paper

Optimal Temporal Logic Planning for Multi-Robot Systems in Uncertain Semantic Maps

  • Yiannis Kantaros
  • George J. Pappas

This paper addresses a multi-robot motion planning problem in probabilistic maps obtained by semantic simultaneous localization and mapping (SLAM). The goal of the robots is to accomplish complex collaborative high level tasks captured by global temporal logic specifications in the presence of uncertainty in the workspace. Specifically, the robots operate in an unknown environment modeled as a semantic map determined by Gaussian distributions over landmark positions and arbitrary discrete distributions over landmark classes. We extend Linear Temporal Logic by including information-based predicates allowing us to incorporate uncertainty and probabilistic satisfaction requirements directly into the task specification. We propose a new highly scalable sampling-based approach that synthesizes paths that satisfy the assigned task specification while minimizing a user-specified motion cost function. Finally, we show that the proposed algorithm is probabilistically complete, asymptotically optimal and supported by convergence rate bounds. We provide extensive simulation results that corroborate the theoretical analysis and show that the proposed algorithm can address large-scale planning tasks.

Details

IJCAI Conference 2018 Conference Paper

A Unifying View of Geometry, Semantics, and Data Association in SLAM

  • Nikolay Atanasov
  • Sean L. Bowman
  • Kostas Daniilidis
  • George J. Pappas

Traditional approaches for simultaneous localization and mapping (SLAM) rely on geometric features such as points, lines, and planes to infer the environment structure. They make hard decisions about the (data) association between observed features and mapped landmarks to update the environment model. This paper makes two contributions to the state of the art in SLAM. First, it generalizes the purely geometric model by introducing semantically meaningful objects, represented as structured models of mid-level part features. Second, instead of making hard, potentially wrong associations between semantic features and objects, it shows that SLAM inference can be performed efficiently with probabilistic data association. The approach not only allows building meaningful maps (containing doors, chairs, cars, etc. ) but also offers significant advantages in ambiguous environments.

PDF Details

IROS Conference 2018 Conference Paper

Resilient Active Information Gathering with Mobile Robots

  • Brent Schlotfeldt
  • Vasileios Tzoumas
  • Dinesh Thakur
  • George J. Pappas

Applications of safety, security, and rescue in robotics, such as multi-robot target tracking, involve the execution of information acquisition tasks by teams of mobile robots. However, in failure-prone or adversarial environments, robots get attacked, their communication channels get jammed, and their sensors may fail, resulting in the withdrawal of robots from the collective task, and consequently the inability of the remaining active robots to coordinate with each other. As a result, traditional design paradigms become insufficient and, in contrast, resilient designs against system-wide failures and attacks become important. In general, resilient design problems are hard, and even though they often involve objective functions that are monotone or submodular, scalable approximation algorithms for their solution have been hitherto unknown. In this paper, we provide the first algorithm, enabling the following capabilities: minimal communication, i. e. , the algorithm is executed by the robots based only on minimal communication between them; system-wide resiliency, i. e. , the algorithm is valid for any number of denial-of-service attacks and failures; and provable approximation performance, i. e. , the algorithm ensures for all monotone (and not necessarily submodular) objective functions a solution that is finitely close to the optimal. We quantify our algorithms approximation performance using a notion of curvature for monotone set functions. We support our theoretical analyses with simulated and real-world experiments, by considering an active information gathering scenario, namely, multi-robot target tracking.

Details

ICRA Conference 2017 Conference Paper

Probabilistic data association for semantic SLAM

  • Sean L. Bowman
  • Nikolay Atanasov 0001
  • Kostas Daniilidis
  • George J. Pappas

Traditional approaches to simultaneous localization and mapping (SLAM) rely on low-level geometric features such as points, lines, and planes. They are unable to assign semantic labels to landmarks observed in the environment. Furthermore, loop closure recognition based on low-level features is often viewpoint-dependent and subject to failure in ambiguous or repetitive environments. On the other hand, object recognition methods can infer landmark classes and scales, resulting in a small set of easily recognizable landmarks, ideal for view-independent unambiguous loop closure. In a map with several objects of the same class, however, a crucial data association problem exists. While data association and recognition are discrete problems usually solved using discrete inference, classical SLAM is a continuous optimization over metric information. In this paper, we formulate an optimization problem over sensor states and semantic landmark positions that integrates metric information, semantic information, and data associations, and decompose it into two interconnected problems: an estimation of discrete data association and landmark class probabilities, and a continuous optimization over the metric states. The estimated landmark and robot poses affect the association and class distributions, which in turn affect the robot-landmark pose optimization. The performance of our algorithm is demonstrated on indoor and outdoor datasets.

Details

IROS Conference 2016 Conference Paper

Online planning for energy-efficient and disturbance-aware UAV operations

  • Nicola Bezzo
  • Kartik Mohta
  • Cameron Nowzari
  • Insup Lee 0001
  • Vijay Kumar 0001
  • George J. Pappas

In this paper we consider an online planning problem for unmanned aerial vehicle (UAV) operations. Specifically, a UAV has the task of reaching a goal from a set of possible goals while minimizing the amount of energy required. Due to unforeseen disturbances, it is possible that initially attractive goals might end up being very expensive during the execution. Thus, two main problems are investigated here: i) how to predict and plan the motion of the UAV at run time to minimize its energy consumption and ii) when to schedule next replanning time to avoid unnecessary periodic re-evaluation executions. Our approach considers a nonlinear model of the system for which a model predictive controller is used to determine the desired control inputs for each possible goal. These control inputs are then used to estimate the energy required to reach the different goals. Finally, a self-triggered scheduling policy determines how long to wait before replanning the goal to aim for. The proposed framework is validated through simulations and experiments in which a quadrotor must choose and reach some goal while being subject to external disturbances.

Details

ICRA Conference 2016 Conference Paper

Optimal temporal logic planning in probabilistic semantic maps

  • Jie Fu 0002
  • Nikolay Atanasov 0001
  • Ufuk Topcu
  • George J. Pappas

This paper considers robot motion planning under temporal logic constraints in probabilistic maps obtained by semantic simultaneous localization and mapping (SLAM). The uncertainty in a map distribution presents a great challenge for obtaining correctness guarantees with respect to the linear temporal logic (LTL) specification. We show that the problem can be formulated as an optimal control problem in which both the semantic map and the logic formula evaluation are stochastic. Our first contribution is to reduce the stochastic control problem for a subclass of LTL to a deterministic shortest path problem by introducing a confidence parameter δ. A robot trajectory obtained from the deterministic problem is guaranteed to have minimum cost and to satisfy the logic specification in the true environment with probability δ. Our second contribution is to design an admissible heuristic function that guides the planning in the deterministic problem towards satisfying the temporal logic specification. This allows us to obtain an optimal and very efficient solution using the A* algorithm. The performance and correctness of our approach are demonstrated in a simulated semantic environment using a differential-drive robot.

Details

ICRA Conference 2015 Conference Paper

Decentralized active information acquisition: Theory and application to multi-robot SLAM

  • Nikolay Atanasov 0001
  • Jerome Le Ny
  • Kostas Daniilidis
  • George J. Pappas

This paper addresses the problem of controlling mobile sensing systems to improve the accuracy and efficiency of gathering information autonomously. It applies to scenarios such as environmental monitoring, search and rescue, surveillance and reconnaissance, and simultaneous localization and mapping (SLAM). A multi-sensor active information acquisition problem, capturing the common characteristics of these scenarios, is formulated. The goal is to design sensor control policies which minimize the entropy of the estimation task, conditioned on the future measurements. First, we provide a non-greedy centralized solution, which is computationally fast, since it exploits linearized sensing models, and memory efficient, since it exploits sparsity in the environment model. Next, we decentralize the control task to obtain linear complexity in the number of sensors and provide suboptimality guarantees. Finally, our algorithms are applied to the multi-robot active SLAM problem to enable a decentralized nonmyopic solution that exploits sparsity in the planning process.

Details

IROS Conference 2014 Conference Paper

Attack resilient state estimation for autonomous robotic systems

  • Nicola Bezzo
  • James Weimer
  • Miroslav Pajic
  • Oleg Sokolsky
  • George J. Pappas
  • Insup Lee 0001

In this paper we present a methodology to control ground robots under malicious attack on sensors. Within the term attack we intend any malicious disturbance injection on sensors, actuators, and controller that would compromise the safety of a robot. In order to guarantee resilience against attacks, we use a control-level technique implemented within a recursive algorithm that takes advantage of redundancy in the information received by the controller. We use the case study of a vehicle cruise-control, however, the strategy we present in this work is general for several applications. Our methodology relays on redundancy in the sensor measurements: specifically we consider N velocity measurements and use a recursive filtering technique that estimates the state of the system while being resilient against sensor attacks by acting on the variance of the measurements noise. Finally, we move our focus on hardware validation demonstrating our algorithm through extensive outdoor experiments conducted on two unmanned ground robots.

Details

IROS Conference 2014 Conference Paper

Automated composition of motion primitives for multi-robot systems from safe LTL specifications

  • Indranil Saha 0001
  • Rattanachai Ramaithitima
  • Vijay Kumar 0001
  • George J. Pappas
  • Sanjit A. Seshia

We present a compositional motion planning framework for multi-robot systems based on an encoding to satisfiability modulo theories (SMT). In our framework, the desired behavior of a group of robots is specified using a set of safe linear temporal logic (LTL) properties. Our method relies on a library of motion primitives, each of which corresponds to a controller that ensures a particular trajectory in a given configuration. Using the closed-loop behavior of the robots under the action of different controllers, we formulate the motion planning problem as an SMT solving problem and use an off-the-shelf SMT solver to generate trajectories for the robots. Our approach can also be extended to synthesize optimal cost trajectories where optimality is defined with respect to the available motion primitives. Experimental results show that our framework can efficiently solve complex motion planning problems in the context of multi-robot systems.

Details

ICRA Conference 2014 Conference Paper

Information acquisition with sensing robots: Algorithms and error bounds

  • Nikolay Atanasov 0001
  • Jerome Le Ny
  • Kostas Daniilidis
  • George J. Pappas

Utilizing the capabilities of configurable sensing systems requires addressing difficult information gathering problems. Near-optimal approaches exist for sensing systems without internal states. However, when it comes to optimizing the trajectories of mobile sensors the solutions are often greedy and rarely provide performance guarantees. Notably, under linear Gaussian assumptions, the problem becomes deterministic and can be solved off-line. Approaches based on submodularity have been applied by ignoring the sensor dynamics and greedily selecting informative locations in the environment. This paper presents a non-greedy algorithm with suboptimality guarantees, which relies on concavity instead of submodularity and takes the sensor dynamics into account. Coupled with linearization and model predictive control, the algorithm can be used to generate adaptive policies for mobile sensors with non-linear sensing models. Applications in gas concentration mapping and target tracking are presented.

Details

ICRA Conference 2013 Conference Paper

Hypothesis testing framework for active object detection

  • Nikolay Atanasov 0001
  • Bharath Sankaran
  • Jerome Le Ny
  • Thomas Koletschka
  • George J. Pappas
  • Kostas Daniilidis

One of the central problems in computer vision is the detection of semantically important objects and the estimation of their pose. Most of the work in object detection has been based on single image processing and its performance is limited by occlusions and ambiguity in appearance and geometry. This paper proposes an active approach to object detection by controlling the point of view of a mobile depth camera. When an initial static detection phase identifies an object of interest, several hypotheses are made about its class and orientation. The sensor then plans a sequence of viewpoints, which balances the amount of energy used to move with the chance of identifying the correct hypothesis. We formulate an active M-ary hypothesis testing problem, which includes sensor mobility, and solve it using a point-based approximate POMDP algorithm. The validity of our approach is verified through simulation and experiments with real scenes captured by a kinect sensor. The results suggest a significant improvement over static object detection.

Details

ICRA Conference 2012 Conference Paper

Sequential composition of robust controller specifications

  • Jerome Le Ny
  • George J. Pappas

We present a general notion of robust controller specification and a mechanism for sequentially composing them. These specifications form tubular abstractions of the trajectories of a system in different control modes, and are motivated by the techniques available for certifying the performance of low-level controllers. The notion of controller specification provides a rigorous interface for connecting a planner and lower-level controllers that are designed independently. With this approach, the planning layer does not integrate the closed-loop system dynamics and does not require the knowledge of how the controllers operate, but relies only on the specifications of the output tracking performance achieved by these controllers. The control layer aims at satisfying specifications that account quantitatively for robustness to unmodeled dynamics and various sources of disturbance and sensor noise, so that this robustness does not need to be revalidated at the planning level. As an illustrative example, we describe a randomized planner that composes different controller specifications from a given database to guarantee that any corresponding sequence of control modes steers a robot to a given region while avoiding obstacles.

Details

ICRA Conference 2012 Conference Paper

Stochastic source seeking in complex environments

  • Nikolay Atanasov 0001
  • Jerome Le Ny
  • Nathan Michael
  • George J. Pappas

The objective of source seeking problems is to determine the minimum of an unknown signal field, which represents a physical quantity of interest, such as heat, chemical concentration, or sound. This paper proposes a strategy for source seeking in a noisy signal field using a mobile robot and based on a stochastic gradient descent algorithm. Our scheme does not require a prior map of the environment or a model of the signal field and is simple enough to be implemented on platforms with limited computational power. We discuss the asymptotic convergence guarantees of algorithm and give specific guidelines for its application to mobile robots in unknown indoor environments with obstacles. Both simulations and real-world experiments were carried out to evaluate the performance of our approach. The results suggest that the algorithm has good finite time performance in complex environments.

Details

ICRA Conference 2011 Conference Paper

Wireless manipulation of single cells using magnetic microtransporters

  • Mahmut Selman Sakar
  • Edward B. Steager
  • Anthony Cowley
  • Vijay Kumar 0001
  • George J. Pappas

For such biomedical applications as single cell manipulation and targeted delivery of chemicals, it is important to fabricate microstructures that can be powered and controlled without a tether in fluidic environments. In this work, we describe the construction and operation of micronsized, biocompatible ferromagnetic microtransporters driven by external magnetic fields capable of exerting forces at the pico Newton scale. We develop microtransporters using a simple, single step micro fabrication technique that allows us to produce large numbers in the same step. We also fabricate microgels to deliver drugs. We demonstrate that the microtransporters can be navigated to separate individual targeted cells with micron-scale precision, and deliver microgels without disturbing the cells in the neighborhood and the local microenvironment.

Details

ICRA Conference 2010 Conference Paper

A duality approach to path planning for multiple robots

  • Nader Motee
  • Ali Jadbabaie
  • George J. Pappas

In this paper, we propose an optimization-based framework for path planning for multiple robots in presence of obstacles. The objective is to find multiple fixed length paths for multiple robots that satisfy the following constraints: (i) bounded curvature, (ii) obstacle avoidance, (iii) and collision avoidance. First, we formulate a relaxation of the path planning problem using polygonal approximations. We show that path planning problem for multiple robots under various constraints and missions, such as curvature and obstacle avoidance constraints as well as rendezvous and maximal total area coverage, can be cast as a nonconvex optimization problem. Then, we propose an alternative dual formulation that results in no duality gap. We show that the alternative dual function can be interpreted as minimum potential energy of a multi-particle system with discontinuous spring-like forces. Finally, we show that using the proposed duality-based framework, an approximation of the minimal length path planning problem (also known as Dubins' problem) in presence of obstacles can be solved efficiently using primal-dual interior-point methods.

Details

ICRA Conference 2010 Conference Paper

Automatic synthesis of robot controllers for tasks with locative prepositions

  • Hadas Kress-Gazit
  • George J. Pappas

This paper describes the synthesis of correct robot control from high-level tasks that include non-projective locative prepositions. Here, locative prepositions such as `near' and `between' are used to refer to regions in the robot's workspace and are part of a high-level task description such as “Always stay near room 1” or “Visit the area between room 2 and room 3”. These prepositions induce a discrete abstraction of the workspace which, together with the rest of the task, is used to synthesize a correct-by-construction robot controller such that the robot is guaranteed to behave as expected, if the task is feasible. This work presents an important step towards allowing linguistic control of robots that is both intuitive and provably correct.

Details

ICRA Conference 2010 Conference Paper

Biosensing and actuation for microbiorobots

  • Mahmut Selman Sakar
  • Edward B. Steager
  • A. Agung Julius
  • Min Jun Kim 0001
  • Vijay Kumar 0001
  • George J. Pappas

In this paper, we describe how signaling networks and actuation in bacterial cells and biomolecular networks of bacteria can be used to develop an integrated micro-bio-robotic system. SU8 microstructures blotted with swarmer cells of Serratia Marcescens in a monolayer are propelled by the bacteria in the absence of any environmental stimulus. We call such microstructures with bacteria Micro Bio Robots (MBRs) and the uncontrolled motion in the absence of stimuli self actuation. Our paper has two primary contributions. First, we demonstrate the control of MBRs using self-actuation, DC electric fields and ultra-violet radiation, and develop experimentally validated mathematical model for the MBRs. This model allows us to use self-actuation and electrokinetic actuation to steer the MBR to any position and orientation in a planar micro channel. Second, we describe the development of biosensors for the MBRs. This is done by attaching genetically engineered Escherichia coli cells that are capable of sensing nonmetabolizable lactose analog methyl-β-D-thiogalactoside (TMG). We describe the fabrication process for MBRs and show experimental results demonstrating sensing, actuation and control.

Details

ICRA Conference 2009 Conference Paper

Harnessing bacterial power in microscale actuation

  • A. Agung Julius
  • Mahmut Selman Sakar
  • Edward B. Steager
  • U. Kei Cheang
  • Min Jun Kim 0001
  • Vijay Kumar 0001
  • George J. Pappas

This paper presents a systematic analysis of the motion of microscale structures actuated by flagellated bacteria. We perform the study both experimentally and theoretically. We use a blotting procedure to attach flagellated bacteria to a buoyancy-neutral plate called a microbarge. The motion of the plate depends on the distribution of the cells on the plate and the stimuli from the environment. We construct a stochastic mathematical model for the system, based on the assumption that the behavior of each bacterium is random and independent of that of its neighbors. The main finding of the paper is that the motion of the barge plus bacteria system is a function of a very small set of parameters. This reduced-dimensional model can be easily estimated using experimental data. We show that the simulation results obtained from the model show an excellent match with the experimentally-observed motion of the barge.

Details

ICRA Conference 2009 Conference Paper

Multi-vehicle path planning in dynamically changing environments

  • Ali Ahmadzadeh
  • Nader Motee
  • Ali Jadbabaie
  • George J. Pappas

In this paper, we propose a path planning method for nonholonomic multi-vehicle system in presence of moving obstacles. The objective is to find multiple fixed length paths for multiple vehicles with the following properties: (i) bounded curvature (ii) obstacle avoidant (iii) collision free. Our approach is based on polygonal approximation of a continuous curve. Using this idea, we formulate an arbitrarily fine relaxation of the path planning problem as a nonconvex feasibility optimization problem. Then, we propound a nonsmooth dynamical systems approach to find feasible solutions of this optimization problem. It is shown that the trajectories of the nonsmooth dynamical system always converge to some equilibria that correspond to the set of feasible solutions of the relaxed problem. The proposed framework can handle more complex mission scenarios for multi-vehicle systems such as rendezvous and area coverage.

Details

TCS Journal 2009 Journal Article

Robustness of temporal logic specifications for continuous-time signals

  • Georgios E. Fainekos
  • George J. Pappas

In this paper, we consider the robust interpretation of Metric Temporal Logic (MTL) formulas over signals that take values in metric spaces. For such signals, which are generated by systems whose states are equipped with non-trivial metrics, for example continuous or hybrid, robustness is not only natural, but also a critical measure of system performance. Thus, we propose multi-valued semantics for MTL formulas, which capture not only the usual Boolean satisfiability of the formula, but also topological information regarding the distance, ε, from unsatisfiability. We prove that any other signal that remains ε -close to the initial one also satisfies the same MTL specification under the usual Boolean semantics. Finally, our framework is applied to the problem of testing formulas of two fragments of MTL, namely Metric Interval Temporal Logic (MITL) and closed Metric Temporal Logic (clMTL), over continuous-time signals using only discrete-time analysis. The motivating idea behind our approach is that if the continuous-time signal fulfills certain conditions and the discrete-time signal robustly satisfies the temporal logic specification, then the corresponding continuous-time signal should also satisfy the same temporal logic specification.

Details DOI

ICRA Conference 2008 Conference Paper

Distributed multi-robot task assignment and formation control

  • Nathan Michael
  • Michael M. Zavlanos
  • Vijay Kumar 0001
  • George J. Pappas

Distributed task assignment for multiple agents raises fundamental and novel problems in control theory and robotics. A new challenge is the development of distributed algorithms that dynamically assign tasks to multiple agents, not relying on a priori assignment information. We address this challenge using market-based coordination protocols where the agents are able to bid for task assignment with the assumption that every agent has knowledge of the maximum number of agents that any given task can accommodate. We show that our approach always achieves the desired assignment of agents to tasks after exploring at most a polynomial number of assignments, dramatically reducing the combinatorial nature of discrete assignment problems. We verify our algorithm through both simulation and experimentation on a team of non-holonomic robots performing distributed formation stabilization and group splitting and merging.

Details

ICRA Conference 2007 Conference Paper

Decidability of Motion Planning with Differential Constraints

  • Peng Cheng 0009
  • George J. Pappas
  • Vijay Kumar 0001

Classical path planning does not address many of the challenges of robotic systems subject to differential constraints. While there have been many recent efforts to develop motion planning algorithms for systems with differential constraints (MPD), very little has been said about the existence of exact algorithms. In other words, the decidability of MPD problems is still an open question. In this paper, we propose a partial answer to this question limiting ourselves to special cases where the trajectory functions of the systems under the finite-dimensional piecewise-continuous controls have a closed-form polynomial formulation. We define an abstract formulation for the MPD problem based on the concept of a control space. We provide an incremental decision algorithm to answer the decidability question and present sufficient conditions for problems to which this algorithm can be applied. Decidability results for several non trivial MPD problems are presented. For example, we show that the question of existence of a trajectory for a Dubin's car with a polygonal rigid body between two specified positions and orientations in a polygonal environment with a fixed and finite number of discontinuities in curvature is decidable.

Details

IROS Conference 2007 Conference Paper

From structured english to robot motion

  • Hadas Kress-Gazit
  • Georgios Fainekos
  • George J. Pappas

Recently, Linear Temporal Logic (LTL) has been successfully applied to high-level task and motion planning problems for mobile robots. One of the main attributes of LTL is its close relationship with fragments of natural language. In this paper, we take the first steps toward building a natural language interface for LTL planning methods with mobile robots as the application domain. For this purpose, we built a structured English language which maps directly to a fragment of LTL.

Details

ICRA Conference 2007 Conference Paper

Leader-Follower Formations: Uncalibrated Vision-Based Localization and Control

  • Gian Luca Mariottini
  • Fabio Morbidi
  • Domenico Prattichizzo
  • George J. Pappas
  • Kostas Daniilidis

This paper focuses on leader-follower formations of mobile robots equipped with panoramic cameras and extend earlier works in the literature addressing both the vision-based localization and control problems. First, a new sufficient analytical condition for localizability is proved and used to shed light on the geometrical meaning of formation localization using uncalibrated vision sensors, here performed with the unscented Kalman filter. Second, we design a feedback control law based on dynamic extension in order to extend the applicability of our control scheme also to the case of distant robots.

Details

ICRA Conference 2007 Conference Paper

Sensor-Based Dynamic Assignment in Distributed Motion Planning

  • Michael M. Zavlanos
  • George J. Pappas

Distributed motion planning of multiple agents raises fundamental and novel problems in control theory and robotics. Recently, one such great challenge has been the development of motion planning algorithms that dynamically assign targets or destinations to multiple homogeneous agents, not relying on any a priori assignment of agents to destinations. In this paper, we address this challenge using two novel ideas. First, we develop distributed multi-destination potential fields able to drive every agent to any available destination for almost all initial conditions. Second, we propose sensor-based coordination protocols that ensure that distinct agents are assigned to distinct destinations. Integration of the overall system results in a distributed, multi-agent, hybrid system for which we show that the mutual exclusion property of the final assignment is guaranteed for almost all initial conditions. Moreover, we show that our dynamic assignment algorithm converges after exploring at most a polynomial number of assignments, dramatically reducing the combinatorial nature of purely discrete assignment problems. Our scalable approach is illustrated with nontrivial computer simulations.

Details

IROS Conference 2007 Conference Paper

Valet parking without a valet

  • David C. Conner
  • Hadas Kress-Gazit
  • Howie Choset
  • Alfred A. Rizzi
  • George J. Pappas

What would it be like if we could give our robot high level commands and it would automatically execute them in a verifiably correct fashion in dynamically changing environments? This work demonstrates a method for generating continuous feedback control inputs that satisfy high-level specifications. Using a collection of continuous local feedback control policies in concert with a synthesized discrete automaton, this paper demonstrates the approach on an Ackermann-steered vehicle that satisfies the command "drive around until you find an empty parking space, then park. " The system reacts to changing environmental conditions using only local information, while guaranteeing the correct high level behavior. The local policies consider the vehicle body shape as well as bounds on drive and steering velocities. The discrete automaton that invokes the local policies guarantees executions that satisfy the high-level specification based only on information about the current availability of the nearest parking space. This paper also demonstrates coordination of two vehicles using the approach.

Details

ICRA Conference 2007 Conference Paper

Where's Waldo? Sensor-Based Temporal Logic Motion Planning

  • Hadas Kress-Gazit
  • Georgios Fainekos
  • George J. Pappas

Given a robot model and a class of admissible environments, this paper provides a framework for automatically and verifiably composing controllers that satisfy high level task specifications expressed in suitable temporal logics. The desired task specifications can express complex robot behaviors such as search and rescue, coverage, and collision avoidance. In addition, our framework explicitly captures sensor specifications that depend on the environment with which the robot is interacting, resulting in a novel paradigm for sensor-based temporal logic motion planning. As one robot is part of the environment of another robot, our sensor-based framework very naturally captures multi-robot specifications. Our computational approach is based on first creating discrete controllers satisfying so-called general reactivity formulas. If feasible, the discrete controller is then used in order to guide the sensor-based composition of continuous controllers resulting in a hybrid controller satisfying the high level specification, but only if the environment is admissible.

Details

TCS Journal 2005 Journal Article

Bisimulation relations for dynamical, control, and hybrid systems

  • Esfandiar Haghverdi
  • Paulo Tabuada
  • George J. Pappas

The fundamental notion of bisimulation equivalence for concurrent processes, has escaped the world of continuous, and subsequently, hybrid systems. Inspired by the categorical framework of Joyal, Nielsen and Winskel, we develop novel notions of bisimulation equivalence for dynamical systems as well as control systems. We prove that these notions can be captured by the abstract notion of bisimulation as developed by Joyal, Nielsen and Winskel. This is the first unified notion of system equivalence that transcends discrete and continuous systems. Furthermore, this enables the development of a novel and natural notion of bisimulation for hybrid systems, which is the final goal of this paper.

Details DOI

ICRA Conference 2005 Conference Paper

Information Driven Coordinated Air-Ground Proactive Sensing

  • Ben Grocholsky
  • Rahul Swaminathan
  • James Keller 0002
  • Vijay Kumar 0001
  • George J. Pappas

This paper concerns the problem of actively searching for and localizing ground features by a coordinated team of air and ground robotic sensor platforms. The approach taken builds on well known Decentralized Data Fusion (DDF) methodology. In particular, it brings together established representations developed for identification and linearized estimation problems to jointly address feature detection and localization. This provides transparent and scalable integration of sensor information from air and ground platforms. As in previous studies, an Information-theoretic utility measure and local control strategy drive the robots to uncertainty reducing team configurations. Complementary characteristics in terms of coverage and accuracy are revealed through analysis of the observation uncertainty for air and ground on-board cameras. Implementation results for a detection and localization example indicate the ability of this approach to scalably and efficiently realize such collaborative potential.

Details

ICRA Conference 2005 Conference Paper

Temporal Logic Motion Planning for Mobile Robots

  • Georgios Fainekos
  • Hadas Kress-Gazit
  • George J. Pappas

In this paper, we consider the problem of robot motion planning in order to satisfy formulas expressible in temporal logics. Temporal logics naturally express traditional robot specifications such as reaching a goal or avoiding an obstacle, but also more sophisticated specifications such as sequencing, coverage, or temporal ordering of different tasks. In order to provide computational solutions to this problem, we first construct discrete abstractions of robot motion based on some environmental decomposition. We then generate discrete plans satisfying the temporal logic formula using powerful model checking tools, and finally translate the discrete plans to continuous trajectories using hybrid control. Critical to our approach is providing formal guarantees ensuring that if the discrete plan satisfies the temporal logic formula, then the continuous motion also satisfies the exact same formula.

Details

IROS Conference 2004 Conference Paper

Hybrid control for visibility-based pursuit-evasion games

  • Volkan Isler
  • Calin Belta
  • Kostas Daniilidis
  • George J. Pappas

Pursuit-evasion games in complex environments have a rich but disconnected history. Continuous or differential pursuit-evasion games focus on optimal control methods, and rely on very intense computations in order to provide locally optimal controls. Discrete pursuit-evasion games on graphs are algorithmically much more appealing, but completely ignore the physical dynamics of the players, resulting in possibly infeasible motions. In this paper, we present a provable and algorithmically feasible solution for visibility-based pursuit-evasion games in simply-connected environments, for players with dynamic constraints. This is achieved by combining two recent but distant results.

Details

TCS Journal 2004 Journal Article

Optimal paths in weighted timed automata

  • Rajeev Alur
  • Salvatore La Torre
  • George J. Pappas

We consider the optimal-reachability problem for a timed automaton with respect to a linear cost function which results in a weighted timed automaton. Our solution to this optimization problem consists of reducing it to computing (parametric) shortest paths in a finite weighted directed graph. We call this graph a parametric sub-region graph. It refines the region graph, a standard tool for the analysis of timed automata, by adding the information which is relevant to solving the optimal-reachability problem. We present an algorithm to solve the optimal-reachability problem for weighted timed automata that takes time exponential in O(n(|δ(A)|+|w max |)), where n is the number of clocks, |δ(A)| is the size of the clock constraints and |w max| is the size of the largest weight. We show that this algorithm can be improved, if we restrict to weighted timed automata with a single clock. In case we consider a single starting state for the optimal-reachability problem, our approach yields an algorithm that takes exponential time only in the length of clock constraints.

Details DOI

ICRA Conference 2002 Conference Paper

The Effect of Feedback and Feedforward on Formation ISS

  • Herbert G. Tanner
  • Vijay Kumar 0001
  • George J. Pappas

A new type of stability of leader follower formations is defined, based on input-to-state stability (ISS) properties of cascade interconnections. Formation ISS links leader input to internal state of the formation and characterizes the way this input affects performance. The effect of feedforward and feedback inter-agent communication is then investigated in this framework and it is indicated how the structure of interconnections and the amount of available information can affect stability performance.

Details

IROS Conference 1997 Conference Paper

Generation of conflict resolution manoeuvres for air traffic management

  • Jana Kosecka
  • Claire J. Tomlin
  • George J. Pappas
  • S. Shankar Sastry

We explore the use of distributed online motion planning algorithms for multiple mobile agents, in air traffic management systems (ATMS). The work is motivated by current trends in ATMS to move towards decentralized air traffic management, in which the aircraft operate in "free flight" mode instead of following prespecified "sky freeways". Conflict resolution strategies are an integral part of the free flight setting. The purpose of this paper is to obtain a set of manoeuvres to cover all possible conflict scenarios involving multiple agents. A distributed motion planning algorithm based on potential and vortex fields is used. While the algorithm is not always guaranteed to generate flyable trajectories, the obtained trajectories can serve as qualitative prototypes for coordination manoeuvres between multiple aircraft. The actual manoeuvres are generated by approximating these prototypes with trajectories made zip of straight lines and are further verified using hybrid verification techniques.

Details
v2026.09.13