Arrow Research search

Author name cluster

Amanda Prorok

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

36 papers
2 author rows

Possible papers

36

AAAI Conference 2026 Conference Paper

Graph Attention-Guided Search for Dense Multi-Agent Pathfinding

  • Rishabh Jain
  • Keisuke Okumura
  • Michael Amir
  • Amanda Prorok

Finding near-optimal solutions for dense multi-agent pathfinding (MAPF) problems in real-time remains challenging even for state-of-the-art planners. To this end, we develop a hybrid framework that integrates a learned heuristic derived from MAGAT, a neural MAPF policy with a graph attention scheme, into a leading search-based algorithm, LaCAM. While prior work has explored learning-guided search in MAPF, such methods have historically underperformed. In contrast, our approach, termed LaGAT, outperforms both purely search-based and purely learning-based methods in dense scenarios. This is achieved through an enhanced MAGAT architecture, a pre-train–then–fine-tune strategy on maps of interest, and a deadlock detection scheme to account for imperfect neural guidance. Our results demonstrate that, when carefully designed, hybrid search offers a powerful solution for tightly coupled, challenging multi-agent coordination problems.

PDF Details DOI

IROS Conference 2025 Conference Paper

D4orm: Multi-Robot Trajectories with Dynamics-aware Diffusion Denoised Deformations

  • Yuhao Zhang
  • Keisuke Okumura 0001
  • Heedo Woo
  • Ajay Shankar
  • Amanda Prorok

This work presents an optimization method for generating kinodynamically feasible and collision-free multi-robot trajectories that exploits an incremental denoising scheme in diffusion models. Our key insight is that high-quality trajectories can be discovered merely by denoising noisy trajectories sampled from a distribution. This approach has no learning component, relying instead on only two ingredients: a dynamical model of the robots to obtain feasible trajectories via rollout, and a fitness function to guide denoising with Monte Carlo gradient approximation. The proposed framework iteratively optimizes a deformation for the previous trajectory with the current denoising process, allows anytime refinement as time permits, supports different dynamics, and benefits from GPU acceleration. Our evaluations for differential-drive and holonomic teams with up to 16 robots in 2D and 3D worlds show its ability to discover high-quality solutions faster than other black-box optimization methods such as MPPI. In a 2D holonomic case with 16 robots, it is almost twice as fast. As evidence for feasibility, we demonstrate zero-shot deployment of the planned trajectories on eight multirotors.

Details

ICRA Conference 2025 Conference Paper

DVM-SLAM: Decentralized Visual Monocular Simultaneous Localization and Mapping for Multi-Agent Systems

  • Joshua Bird
  • Jan Blumenkamp
  • Amanda Prorok

Cooperative Simultaneous Localization and Mapping (C-SLAM) enables multiple agents to work together in mapping unknown environments while simultaneously estimating their own positions. This approach enhances robustness, scalability, and accuracy by sharing information between agents, reducing drift, and enabling collective exploration of larger areas. In this paper, we present Decentralized Visual Monocular SLAM (DVM-SLAM), the first open-source decentralized monocular C-SLAM system. By only utilizing low-cost and light-weight monocular vision sensors, our system is well suited for small robots and micro aerial vehicles (MAVs). DVMSLAM's real-world applicability is validated on physical robots with a custom collision avoidance framework, showcasing its potential in real-time multi-agent autonomous navigation scenarios. We also demonstrate comparable accuracy to state-of-the-art centralized monocular C-SLAM systems. We opensource our code and provide supplementary material online ${ }^{1}$.

Details

ICLR Conference 2025 Conference Paper

Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling

  • Jasmine Bayrooti
  • Carl Henrik Ek
  • Amanda Prorok

Learning complex robot behavior through interactions with the environment necessitates principled exploration. Effective strategies should prioritize exploring regions of the state-action space that maximize rewards, with optimistic exploration emerging as a promising direction aligned with this idea and enabling sample-efficient reinforcement learning. However, existing methods overlook a crucial aspect: the need for optimism to be informed by a belief connecting the reward and state. To address this, we propose a practical, theoretically grounded approach to optimistic exploration based on Thompson sampling. Our approach is the first that allows for reasoning about _joint_ uncertainty over transitions and rewards for optimistic exploration. We apply our method on a set of MuJoCo and VMAS continuous control tasks. Our experiments demonstrate that optimistic exploration significantly accelerates learning in environments with sparse rewards, action penalties, and difficult-to-explore regions. Furthermore, we provide insights into when optimism is beneficial and emphasize the critical role of model uncertainty in guiding exploration.

Details

ICRA Conference 2025 Conference Paper

Language-Conditioned Offline RL for Multi-Robot Navigation

  • Steven D. Morad
  • Ajay Shankar
  • Jan Blumenkamp
  • Amanda Prorok

We present a method for synthesizing navigation policies for multi-robot teams that interpret and follow natural language instructions. We condition these policies on embeddings from pretrained Large Language Models (LLMs), and train them via offline reinforcement learning with as little as 20 minutes of randomly-collected real-world data. Experiments on a team of five real robots show that these policies generalize well to unseen commands, indicating an understanding of the LLM latent space. Our method requires no simulators or environment models, and produces low-latency control policies that can be deployed directly to real robots without finetuning. We provide videos of our experiments at https://sites.google.com/view/llm-marl.

Details

NeurIPS Conference 2025 Conference Paper

No-Regret Thompson Sampling for Finite-Horizon Markov Decision Processes with Gaussian Processes

  • Jasmine Bayrooti
  • Sattar Vakili
  • Amanda Prorok
  • Carl Henrik Ek

Thompson sampling (TS) is a powerful and widely used strategy for sequential decision-making, with applications ranging from Bayesian optimization to reinforcement learning (RL). Despite its success, the theoretical foundations of TS remain limited, particularly in settings with complex temporal structure such as RL. We address this gap by establishing no-regret guarantees for TS using models with Gaussian marginal distributions. Specifically, we consider TS in episodic RL with joint Gaussian process (GP) priors over rewards and transitions. We prove a regret bound of $\mathcal{\tilde{O}}(\sqrt{KH\Gamma(KH)})$ over $K$ episodes of horizon $H$, where $\Gamma(\cdot)$ captures the complexity of the GP model. Our analysis addresses several challenges, including the non-Gaussian nature of value functions and the recursive structure of Bellman updates, and extends classical tools such as the elliptical potential lemma to multi-output settings. This work advances the understanding of TS in RL and highlights how structural assumptions and model uncertainty shape its performance in finite-horizon Markov Decision Processes.

PDF Details

JMLR Journal 2025 Journal Article

System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning

  • Matteo Bettini
  • Ajay Shankar
  • Amanda Prorok

Evolutionary science provides evidence that diversity confers resilience in natural systems. Yet, traditional multi-agent reinforcement learning techniques commonly enforce homogeneity to increase training sample efficiency. When a system of learning agents is not constrained to homogeneous policies, individuals may develop diverse behaviors, resulting in emergent complementarity that benefits the system. Despite this, there is a surprising lack of tools that quantify behavioral diversity. Such techniques would pave the way towards understanding the impact of diversity in collective artificial intelligence and enabling its control. In this paper, we introduce System Neural Diversity (SND): a measure of behavioral heterogeneity in multi-agent systems. We discuss and prove its theoretical properties, and compare it with alternate, state-of-the-art behavioral diversity metrics used in the robotics domain. Through simulations of a variety of cooperative multi-robot tasks, we show how our metric constitutes an important tool that enables measurement and control of behavioral heterogeneity. In dynamic tasks, where the problem is affected by repeated disturbances during training, we show that SND allows us to measure latent resilience skills acquired by the agents, while other proxies, such as task performance (reward), fail to. Finally, we show how the metric can be employed to control diversity, allowing us to enforce a desired heterogeneity set-point or range. We demonstrate how this paradigm can be used to bootstrap the exploration phase, finding optimal policies faster, thus enabling novel and more efficient MARL paradigms. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2025. ( edit, beta )

PDF Details

JMLR Journal 2024 Journal Article

BenchMARL: Benchmarking Multi-Agent Reinforcement Learning

  • Matteo Bettini
  • Amanda Prorok
  • Vincent Moens

The field of Multi-Agent Reinforcement Learning (MARL) is currently facing a reproducibility crisis. While solutions for standardized reporting have been proposed to address the issue, we still lack a benchmarking tool that enables standardization and reproducibility, while leveraging cutting-edge Reinforcement Learning (RL) implementations. In this paper, we introduce BenchMARL, the first MARL training library created to enable standardized benchmarking across different algorithms, models, and environments. BenchMARL uses TorchRL as its backend, granting it high-performance and maintained state-of-the-art implementations while addressing the broad community of MARL PyTorch users. Its design enables systematic configuration and reporting, thus allowing users to create and run complex benchmarks from simple one-line inputs. BenchMARL is open-sourced on GitHub at https://github.com/facebookresearch/BenchMARL [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2024. ( edit, beta )

PDF Details

ICML Conference 2024 Conference Paper

Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning

  • Matteo Bettini
  • Ryan Kortvelesy
  • Amanda Prorok

The study of behavioral diversity in Multi-Agent Reinforcement Learning (MARL) is a nascent yet promising field. In this context, the present work deals with the question of how to control the diversity of a multi-agent system. With no existing approaches to control diversity to a set value, current solutions focus on blindly promoting it via intrinsic rewards or additional loss functions, effectively changing the learning objective and lacking a principled measure for it. To address this, we introduce Diversity Control (DiCo), a method able to control diversity to an exact value of a given metric by representing policies as the sum of a parameter-shared component and dynamically scaled per-agent components. By applying constraints directly to the policy architecture, DiCo leaves the learning objective unchanged, enabling its applicability to any actor-critic MARL algorithm. We theoretically prove that DiCo achieves the desired diversity, and we provide several experiments, both in cooperative and competitive tasks, that show how DiCo can be employed as a novel paradigm to increase performance and sample efficiency in MARL. Multimedia results are available on the paper’s website: https: //sites. google. com/view/dico-marl

Details

NeurIPS Conference 2024 Conference Paper

Recurrent Reinforcement Learning with Memoroids

  • Steven Morad
  • Chris Lu
  • Ryan Kortvelesy
  • Stephan Liwicki
  • Jakob Foerster
  • Amanda Prorok

Memory models such as Recurrent Neural Networks (RNNs) and Transformers address Partially Observable Markov Decision Processes (POMDPs) by mapping trajectories to latent Markov states. Neither model scales particularly well to long sequences, especially compared to an emerging class of memory models called Linear Recurrent Models. We discover that the recurrent update of these models resembles a monoid, leading us to reformulate existing models using a novel monoid-based framework that we call memoroids. We revisit the traditional approach to batching in recurrent reinforcement learning, highlighting theoretical and empirical deficiencies. We leverage memoroids to propose a batching method that improves sample efficiency, increases the return, and simplifies the implementation of recurrent loss functions in reinforcement learning.

PDF Details DOI

ICRA Conference 2023 Conference Paper

Accelerating Multi-Agent Planning Using Graph Transformers with Bounded Suboptimality

  • Chenning Yu
  • Qingbiao Li
  • Sicun Gao
  • Amanda Prorok

Conflict-Based Search is one of the most popular methods for multi-agent path finding. Though it is complete and optimal, it does not scale well. Recent works have been proposed to accelerate it by introducing various heuristics. However, whether these heuristics can apply to non-grid-based problem settings while maintaining their effectiveness remains an open question. In this work, we find that the answer is prone to be no. To this end, we propose a learning-based component, i. e. , the Graph Transformer, as a heuristic function to accelerate the planning. The proposed method is provably complete and bounded-suboptimal with any desired factor. We conduct extensive experiments on two environments with dense graphs. Results show that the proposed Graph Transformer can be trained in problem instances with relatively few agents and generalizes well to a larger number of agents, while achieving better performance than state-of-the-art methods.

Details

ICRA Conference 2023 Conference Paper

Environment Optimization for Multi-Agent Navigation

  • Zhan Gao
  • Amanda Prorok

Traditional approaches to the design of multiagent navigation algorithms consider the environment as a fixed constraint, despite the obvious influence of spatial constraints on agents' performance. Yet hand-designing improved environment layouts and structures is inefficient and potentially expensive. The goal of this paper is to consider the environment as a decision variable in a system-level optimization problem, where both agent performance and environment cost can be accounted for. We begin by proposing a novel environment optimization problem. We show, through formal proofs, under which conditions the environment can change while guaranteeing completeness (i. e. , all agents reach their navigation goals). Our solution leverages a model-free reinforcement learning approach. In order to accommodate a broad range of implementation scenarios, we include both online and offline optimization, and both discrete and continuous environment representations. Numerical results corroborate our theoretical findings and validate our approach.

Details

NeurIPS Conference 2023 Conference Paper

Generalised f-Mean Aggregation for Graph Neural Networks

  • Ryan Kortvelesy
  • Steven Morad
  • Amanda Prorok

Graph Neural Network (GNN) architectures are defined by their implementations of update and aggregation modules. While many works focus on new ways to parametrise the update modules, the aggregation modules receive comparatively little attention. Because it is difficult to parametrise aggregation functions, currently most methods select a ``standard aggregator'' such as mean, sum, or max. While this selection is often made without any reasoning, it has been shown that the choice in aggregator has a significant impact on performance, and the best choice in aggregator is problem-dependent. Since aggregation is a lossy operation, it is crucial to select the most appropriate aggregator in order to minimise information loss. In this paper, we present GenAgg, a generalised aggregation operator, which parametrises a function space that includes all standard aggregators. In our experiments, we show that GenAgg is able to represent the standard aggregators with much higher accuracy than baseline methods. We also show that using GenAgg as a drop-in replacement for an existing aggregator in a GNN often leads to a significant boost in performance across various tasks.

PDF Details

AAMAS Conference 2023 Conference Paper

Heterogeneous Multi-Robot Reinforcement Learning

  • Matteo Bettini
  • Ajay Shankar
  • Amanda Prorok

Cooperative multi-robot tasks can benefit from heterogeneity in the robots’ physical and behavioral traits. In spite of this, traditional Multi-Agent Reinforcement Learning (MARL) frameworks lack the ability to explicitly accommodate policy heterogeneity, and typically constrain agents to share neural network parameters. This enforced homogeneity limits application in cases where the tasks benefit from heterogeneous behaviors. In this paper, we crystallize the role of heterogeneity in MARL policies. Towards this end, we introduce Heterogeneous Graph Neural Network Proximal Policy Optimization (HetGPPO), a paradigm for training heterogeneous MARL policies that leverages a Graph Neural Network for differentiable inter-agent communication. HetGPPO allows communicating agents to learn heterogeneous behaviors while enabling fully decentralized training in partially observable environments. We complement this with a taxonomical overview that exposes more heterogeneity classes than previously identified. To motivate the need for our model, we present a characterization of techniques that homogeneous models can leverage to emulate heterogeneous behavior, and show how this “apparent heterogeneity” is brittle in real-world conditions. Through simulations and real-world experiments, we show that: (i) when homogeneous methods fail due to strong heterogeneous requirements, HetGPPO succeeds, and, (ii) when homogeneous methods are able to learn apparently heterogeneous behaviors, HetGPPO achieves higher resilience to both training and deployment noise.

PDF

AAMAS Conference 2023 Conference Paper

Permutation-Invariant Set Autoencoders with Fixed-Size Embeddings for Multi-Agent Learning

  • Ryan Kortvelesy
  • Steven Morad
  • Amanda Prorok

The problem of permutation-invariant learning over set representations is particularly relevant in the field of multi-agent systems—a few potential applications include unsupervised training of aggregation functions in graph neural networks (GNNs), neural cellular automata on graphs, and prediction of scenes with multiple objects. Yet existing approaches to set encoding and decoding tasks present a host of issues, including non-permutation-invariance, fixed-length outputs, reliance on iterative methods, non-deterministic outputs, computationally expensive loss functions, and poor reconstruction accuracy. In this paper we introduce a Permutation-Invariant Set Autoencoder (PISA), which tackles these problems and produces encodings with significantly lower reconstruction error than existing baselines. PISA also provides other desirable properties, including a similarity-preserving latent space, and the ability to insert or remove elements from the encoding. After evaluating PISA against baseline methods, we demonstrate its usefulness in a multi-agent application. Using PISA as a subcomponent, we introduce a novel GNN architecture which serves as a generalised communication scheme, allowing agents to use communication to gain full observability of a system.

PDF

ICLR Conference 2023 Conference Paper

POPGym: Benchmarking Partially Observable Reinforcement Learning

  • Steven D. Morad
  • Ryan Kortvelesy
  • Matteo Bettini
  • Stephan Liwicki
  • Amanda Prorok

Real world applications of Reinforcement Learning (RL) are often partially observable, thus requiring memory. Despite this, partial observability is still largely ignored by contemporary RL benchmarks and libraries. We introduce Partially Observable Process Gym (POPGym), a two-part library containing (1) a diverse collection of 15 partially observable environments, each with multiple difficulties and (2) implementations of 13 memory model baselines -- the most in a single RL library. Existing partially observable benchmarks tend to fixate on 3D visual navigation, which is computationally expensive and only one type of POMDP. In contrast, POPGym environments are diverse, produce smaller observations, use less memory, and often converge within two hours of training on a consumer-grade GPU. We implement our high-level memory API and memory baselines on top of the popular RLlib framework, providing plug-and-play compatibility with various training algorithms, exploration strategies, and distributed training paradigms. Using POPGym, we execute the largest comparison across RL memory models to date. POPGym is available at https://github.com/proroklab/popgym.

Details

NeurIPS Conference 2023 Conference Paper

Reinforcement Learning with Fast and Forgetful Memory

  • Steven Morad
  • Ryan Kortvelesy
  • Stephan Liwicki
  • Amanda Prorok

Nearly all real world tasks are inherently partially observable, necessitating the use of memory in Reinforcement Learning (RL). Most model-free approaches summarize the trajectory into a latent Markov state using memory models borrowed from Supervised Learning (SL), even though RL tends to exhibit different training and efficiency characteristics. Addressing this discrepancy, we introduce Fast and Forgetful Memory, an algorithm-agnostic memory model designed specifically for RL. Our approach constrains the model search space via strong structural priors inspired by computational psychology. It is a drop-in replacement for recurrent neural networks (RNNs) in recurrent RL algorithms, achieving greater reward than RNNs across various recurrent benchmarks and algorithms without changing any hyperparameters. Moreover, Fast and Forgetful Memory exhibits training speeds two orders of magnitude faster than RNNs, attributed to its logarithmic time and linear space complexity. Our implementation is available at https: //github. com/proroklab/ffm.

PDF Details

IROS Conference 2023 Conference Paper

See What the Robot Can't See: Learning Cooperative Perception for Visual Navigation

  • Jan Blumenkamp
  • Qingbiao Li
  • Binyu Wang
  • Zhe Liu 0022
  • Amanda Prorok

We consider the problem of navigating a mobile robot towards a target in an unknown environment that is endowed with visual sensors, where neither the robot nor the sensors have access to global positioning information and only use first-person- view images. In order to overcome the need for positioning, we train the sensors to encode and communicate relevant viewpoint information to the mobile robot, whose objective it is to use this information to navigate to the target along the shortest path. We overcome the challenge of enabling all the sensors (even those that cannot directly see the target) to predict the direction along the shortest path to the target by implementing a neighborhood-based feature aggregation module using a Graph Neural Network (GNN) architecture. In our experiments, we first demonstrate generalizability to previously unseen environments with various sensor layouts. Our results show that by using communication between the sensors and the robot, we achieve up to 2. 0 × improvement in SPL (Success weighted by Path Length) when compared to a communication-free baseline. This is done without requiring a global map, positioning data, nor pre-calibration of the sensor network. Second, we perform a zero-shot transfer of our model from simulation to the real world. Laboratory experiments demonstrate the feasibility of our approach in various cluttered environments. Finally, we showcase examples of successful navigation to the target while both the sensor network layout as well as obstacles are dynamically reconfigured as the robot navigates. We provide a video demo 1 1 https://www.youtube.com/watch?v=kcrnr6RUgucw, the dataset, trained models, and source code 2 2 https://github.com/proroklab/sensor-guided-visual-nav.

Details

ICRA Conference 2022 Conference Paper

A Framework for Real-World Multi-Robot Systems Running Decentralized GNN-Based Policies

  • Jan Blumenkamp
  • Steven D. Morad
  • Jennifer Gielis
  • Qingbiao Li
  • Amanda Prorok

Graph Neural Networks (GNNs) are a paradigm-shifting neural architecture to facilitate the learning of complex multi-agent behaviors. Recent work has demonstrated remarkable performance in tasks such as flocking, multi-agent path planning and cooperative coverage. However, the policies derived through GNN-based learning schemes have not yet been deployed to the real-world on physical multi-robot systems. In this work, we present the design of a system that allows for fully decentralized execution of GNN-based policies. We create a framework based on ROS2 and elaborate its details in this paper. We demonstrate our framework on a case-study that requires tight coordination between robots, and present first-of-a-kind results that show successful real-world deployment of GNN-based policies on a decentralized multi-robot system relying on Adhoc communication. A video demonstration of this case-study can be found online 1 1 youtube.com/watch?v=COh-WLn4i04.

Details

ICLR Conference 2022 Conference Paper

Graph Neural Network Guided Local Search for the Traveling Salesperson Problem

  • Benjamin Hudson
  • Qingbiao Li
  • Matthew Malencia
  • Amanda Prorok

Solutions to the Traveling Salesperson Problem (TSP) have practical applications to processes in transportation, logistics, and automation, yet must be computed with minimal delay to satisfy the real-time nature of the underlying tasks. However, solving large TSP instances quickly without sacrificing solution quality remains challenging for current approximate algorithms. To close this gap, we present a hybrid data-driven approach for solving the TSP based on Graph Neural Networks (GNNs) and Guided Local Search (GLS). Our model predicts the regret of including each edge of the problem graph in the solution; GLS uses these predictions in conjunction with the original problem graph to find solutions. Our experiments demonstrate that this approach converges to optimal solutions at a faster rate than three recent learning based approaches for the TSP. Notably, we reduce the mean optimality gap on the 100-node problem set from 1.534% to 0.705%, a 2x improvement. When generalizing from 20-node instances to the 100-node problem set, we reduce the optimality gap from 18.845% to 2.622%, a 7x improvement.

Details

AAMAS Conference 2022 Conference Paper

The Holy Grail of Multi-Robot Planning: Learning to Generate Online-Scalable Solutions from Offline-Optimal Experts

  • Amanda Prorok
  • Jan Blumenkamp
  • Qingbiao Li
  • Ryan Kortvelesy
  • Zhe Liu
  • Ethan Stump

Many multi-robot planning problems are burdened by the curse of dimensionality, which compounds the difficulty of applying solutions to large-scale problem instances. The use of learning-based methods in multi-robot planning holds great promise as it enables us to offload the online computational burden of expensive centralized, yet optimal solvers, to an offline learning procedure. The hope is that by training a policy to copy an optimal pattern generated by a small-scale (centralized) system, we can transfer that policy to much larger, decentralized systems while maintaining near-optimal performance. Yet, a number of issues impede us from leveraging this idea to its full potential. This blue-sky paper elaborates some of the key challenges that remain.

PDF

IROS Conference 2021 Conference Paper

Learning to Fly - a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control

  • Jacopo Panerati
  • Hehui Zheng
  • Siqi Zhou
  • James Xu
  • Amanda Prorok
  • Angela P. Schoellig

Robotic simulators are crucial for academic research and education as well as the development of safety-critical applications. Reinforcement learning environments— simple simulations coupled with a problem specification in the form of a reward function—are also important to standardize the development (and benchmarking) of learning algorithms. Yet, full-scale simulators typically lack portability and paral-lelizability. Vice versa, many reinforcement learning environments trade-off realism for high sample throughputs in toy-like problems. While public data sets have greatly benefited deep learning and computer vision, we still lack the software tools to simultaneously develop—and fairly compare—control theory and reinforcement learning approaches. In this paper, we propose an open-source OpenAI Gym-like environment for multiple quadcopters based on the Bullet physics engine. Its multi-agent and vision-based reinforcement learning interfaces, as well as the support of realistic collisions and aerodynamic effects, make it, to the best of our knowledge, a first of its kind. We demonstrate its use through several examples, either for control (trajectory tracking with PID control, multi-robot flight with downwash, etc.) or reinforcement learning (single and multi-agent stabilization tasks), hoping to inspire future research that combines control theory and machine learning.

Details

ICRA Conference 2021 Conference Paper

ModGNN: Expert Policy Approximation in Multi-Agent Systems with a Modular Graph Neural Network Architecture

  • Ryan Kortvelesy
  • Amanda Prorok

Recent work in the multi-agent domain has shown the promise of Graph Neural Networks (GNNs) to learn complex coordination strategies. However, most current approaches use minor variants of a Graph Convolutional Network (GCN), which applies a convolution to the communication graph formed by the multi-agent system. In this paper, we investigate whether the performance and generalization of GCNs can be improved upon. We introduce ModGNN, a decentralized framework which serves as a generalization of GCNs, providing more flexibility. To test our hypothesis, we evaluate an implementation of ModGNN against several baselines in the multi-agent flocking problem. We perform an ablation analysis to show that the most important component of our framework is one that does not exist in a GCN. By varying the number of agents, we also demonstrate that an application-agnostic implementation of ModGNN possesses an improved ability to generalize to new environments.

Details

ICRA Conference 2020 Conference Paper

DC-CAPT: Concurrent Assignment and Planning of Trajectories for Dubins Cars

  • Michael Whitzer
  • Daigo Shishika
  • Dinesh Thakur
  • Vijay Kumar 0001
  • Amanda Prorok

We present an algorithm for the concurrent assignment and planning of collision-free trajectories (DC-CAPT) for robots whose kinematics can be modeled as Dubins cars, i. e. , robots constrained in terms of their initial orientation and their minimum turning radius. Coupling the assignment and trajectory planning subproblems allows for a computationally tractable solution. This solution is guaranteed to be collision- free through the use of a single constraint: the start and goal locations have separation distance greater than some threshold. We derive this separation distance by extending a prior work that assumed holonomic robots. We demonstrate the validity of our approach, and show its efficacy through simulations and experiments where groups of robots executing Dubins curves travel to their assigned goal locations without collisions.

Details

IROS Conference 2020 Conference Paper

Graph Neural Networks for Decentralized Multi-Robot Path Planning

  • Qingbiao Li
  • Fernando Gama
  • Alejandro Ribeiro
  • Amanda Prorok

Effective communication is key to successful, decentralized, multi-robot path planning. Yet, it is far from obvious what information is crucial to the task at hand, and how and when it must be shared among robots. To side-step these issues and move beyond hand-crafted heuristics, we propose a combined model that automatically synthesizes local communication and decision-making policies for robots navigating in constrained workspaces. Our architecture is composed of a convolutional neural network (CNN) that extracts adequate features from local observations, and a graph neural network (GNN) that communicates these features among robots. We train the model to imitate an expert algorithm, and use the resulting model online in decentralized planning involving only local communication and local observations. We evaluate our method in simulations by navigating teams of robots to their destinations in 2D cluttered workspaces. We measure the success rates and sum of costs over the planned paths. The results show a performance close to that of our expert algorithm, demonstrating the validity of our approach. In particular, we show our model's capability to generalize to previously unseen cases (involving larger environments and larger robot teams).

Details

ICRA Conference 2020 Conference Paper

Multi-Robot Path Deconfliction through Prioritization by Path Prospects

  • Wenying Wu
  • Subhrajit Bhattacharya
  • Amanda Prorok

This work deals with the problem of planning conflict-free paths for mobile robots in cluttered environments. Since centralized, coupled planning algorithms are computationally intractable for large numbers of robots, we consider decoupled planning, in which robots plan their paths sequentially in order of priority. Choosing how to prioritize the robots is a key consideration. State-of-the-art prioritization heuristics, however, do not model the coupling between a robot's mobility and its environment. This is particularly relevant when prioritizing between robots with different degrees of mobility. In this paper, we propose a prioritization rule that can be computed online by each robot independently, and that provides consistent, conflict-free path plans. Our innovation is to formalize a robot's path prospects to reach its goal from its current location. To this end, we consider the number of homology classes of trajectories, which capture distinct prospects of paths for each robot. This measure is used as a prioritization rule, whenever any robots enter negotiation to deconflict path plans. We perform simulations with heterogeneous robot teams and compare our method to five benchmarks. Our method achieves the highest success rate, and strikes a good balance between makespan and flowtime objectives.

Details

ICRA Conference 2019 Conference Paper

A Fleet of Miniature Cars for Experiments in Cooperative Driving

  • Nicholas Hyldmar
  • Yijun He
  • Amanda Prorok

We introduce a unique experimental testbed that consists of a fleet of 16 miniature Ackermann-steering vehicles. We are motivated by a lack of available low-cost platforms to support research and education in multi-car navigation and trajectory planning. This article elaborates the design of our miniature robotic car, the Cambridge Minicar, as well as the fleet's control architecture. Our experimental testbed allows us to implement state-of-the-art driver models as well as autonomous control strategies, and test their validity in a real, physical multi-lane setup. Through experiments on our miniature highway, we are able to tangibly demonstrate the benefits of cooperative driving on multi-lane road topographies. Our setup paves the way for indoor large-fleet experimental research.

Details

IROS Conference 2017 Conference Paper

A decentralized algorithm for assembling structures with modular robots

  • David Saldaña
  • Bruno Gabrich
  • Michael Whitzer
  • Amanda Prorok
  • Mario F. M. Campos
  • Mark Yim
  • Vijay Kumar 0001

Recent work in the field of bio-inspired robotic systems has introduced designs for modular robots that are able to assemble into structures (e. g. , bridges, landing platforms, fences) using their bodies as the building components. Yet, it remains an open question as to how to program large swarms of robotic modules so that the assembly task is performed as efficiently as possible. Moreover, the problem of designing assembly algorithms is compounded by the scale of these systems, and by the lack of centralized guidance in unstructured environments. The main contribution of this work is a decentralized algorithm to assemble structures with modular robots. Importantly, we coordinate the robots so that docking actions can be parallelized. We show the correctness of our algorithm, and we demonstrate its scalability and generality through multiple scenarios in simulation. Experiments on physical robots demonstrate the validity of our approach in real-world settings.

Details

IROS Conference 2017 Conference Paper

Intrusion detection for stochastic task allocation in robot swarms

  • Florian Maushart
  • Amanda Prorok
  • M. Ani Hsieh
  • Vijay Kumar 0001

We present a novel framework for integrity analysis of swarm robotic systems using the symmetric Kullback-Leibler Divergence. The objective is to understand a robot swarm's vulnerability to malicious intrusion and to develop the necessary computational tools that would detect the presence of malicious agents within the swarm. Using ensemble approaches for modeling and analyzing stochastic task allocation, we analyze the performance of the proposed strategy subject to different system parameters, and show how different design choices can facilitate early intrusion detection. We further evaluate the performance of our method in realistic scenarios through stochastic simulations for different team sizes. The main contribution is an analysis framework whose output can be used to avoid system-inherent design flaws and to decrease the damage that can be inflicted by an undetected attacker.

Details

IROS Conference 2017 Conference Paper

Privacy-preserving vehicle assignment for mobility-on-demand systems

  • Amanda Prorok
  • Vijay Kumar 0001

Urban transportation is being transformed by mobility-on-demand (MoD) systems. One of the goals of MoD systems is to provide personalized transportation services to passengers. This process is facilitated by a centralized operator that coordinates the assignment of vehicles to individual passengers, based on location data. However, current approaches assume that accurate positioning information for passengers and vehicles is readily available. This assumption raises privacy concerns. In this work, we address this issue by proposing a method that protects passengers' drop-off locations (i. e. , their travel destinations). Formally, we solve a batch assignment problem that routes vehicles at obfuscated origin locations to passenger locations (since origin locations correspond to previous drop-off locations), such that the mean waiting time is minimized. Our main contributions are two-fold. First, we formalize the notion of privacy for continuous vehicle-to-passenger assignment in MoD systems, and integrate a privacy mechanism that provides formal guarantees. Second, we present a polynomial-time iterative version of the Hungarian algorithm to allocate a redundant number of vehicles to a single passenger. This algorithm builds on the insight that even during peak rush hour there are unoccupied (redundant) traveling vehicles. This strategy allows us to reduce the performance deterioration induced by the privacy mechanism. In particular, it enables the exploration of the trade-off between privacy levels, waiting time, and deployed fleet size. We evaluate our methods on a real, large-scale data set consisting of over 11 million taxi rides (specifying vehicle availability and passenger requests), recorded over a month's duration, in the area of Manhattan, New York. Based on current traffic statistics, our evaluations indicate that privacy can be achieved without incurring a significant loss of performance, and that this loss can be further controlled by varying operator or user preferences.

Details

ICRA Conference 2016 Conference Paper

Formalizing the impact of diversity on performance in a heterogeneous swarm of robots

  • Amanda Prorok
  • M. Ani Hsieh
  • Vijay Kumar 0001

We are interested in a principled study of the impact of diversity in heterogeneous large-scale distributed robotic systems. In order to evaluate the implications of heterogeneity on performance, we consider the concrete problem of distributing a large group of robots among a set of tasks that require specialized capabilities in order to be completed. We model the system of heterogeneous robots as a community of species, where each species (robot type) is defined by the traits (capabilities) that it owns. We develop a continuous model of the system at a macroscopic level, and formulate an optimization problem that produces an optimal set of transition rates for each species, so that the desired trait distribution is reached as quickly as possible. In order to evaluate the effects of heterogeneity, we propose a diversity metric that defines the notion of eigenspecies. We show that our metric correlates with performance: the higher the cardinality of the eigenspecies, the harder it becomes to optimize the system. Our approach is validated over multiple levels of abstraction, and real robot results confirm its validity on physical platforms.

Details

ICRA Conference 2012 Conference Paper

Low-cost collaborative localization for large-scale multi-robot systems

  • Amanda Prorok
  • Alexander Bahr
  • Alcherio Martinoli

Large numbers of collaborating robots are advantageous for solving distributed problems. In order to efficiently solve the task at hand, the robots often need accurate localization. In this work, we address the localization problem by developing a solution that has low computational and sensing requirements, and that is easily deployed on large robot teams composed of cheap robots. We build upon a real-time, particle-filter based localization algorithm that is completely decentralized and scalable, and accommodates realistic robot assumptions including noisy sensors, and asynchronous and lossy communication. In order to further reduce this algorithm's overall complexity, we propose a low-cost particle clustering method, which is particularly well suited to the collaborative localization problem. Our approach is experimentally validated on a team of ten real robots.

Details

ICRA Conference 2012 Conference Paper

Online model estimation of ultra-wideband TDOA measurements for mobile robot localization

  • Amanda Prorok
  • Lukas Gonon
  • Alcherio Martinoli

Ultra-wideband (UWB) localization is a recent technology that promises to outperform many indoor localization methods currently available. Yet, non-line-of-sight (NLOS) positioning scenarios can create large biases in the time-difference-of-arrival (TDOA) measurements, and must be addressed with accurate measurement models in order to avoid significant localization errors. In this work, we first develop an efficient, closed-form TDOA error model and analyze its estimation characteristics by calculating the Cramér-Rao lower bound (CRLB). We subsequently detail how an online Expectation Maximization (EM) algorithm is adopted to find an elegant formalism for the maximum likelihood estimate of the model parameters. We perform real experiments on a mobile robot equipped with an UWB emitter, and show that the online estimation algorithm leads to excellent localization performance due to its ability to adapt to the varying NLOS path conditions over time.

Details

IROS Conference 2011 Conference Paper

A reciprocal sampling algorithm for lightweight distributed multi-robot localization

  • Amanda Prorok
  • Alcherio Martinoli

This work is situated in the context of collaboratively solving the localization problem for unknown initial conditions. We address this problem with a novel, fully decentralized, real-time particle filter algorithm, designed to accommodate realistic robotic assumptions including noisy sensors, and asynchronous and lossy communication. In particular, we introduce a collaborative reciprocal sampling algorithm which allows a drastic reduction in the number of particles needed to achieve localization. We elaborate an analysis of our reciprocal sampling method and support our conclusions with simulation results. Finally, we validate our approach on a team of four real robots within a controlled experimental setup.

Details

IROS Conference 2011 Conference Paper

Two-phase online calibration for infrared-based inter-robot positioning modules

  • Sven Gowal
  • Amanda Prorok
  • Alcherio Martinoli

Multi-robot systems can solve complex tasks that require the coordination of the team-member positions with respect to each other. While the development of ad-hoc relative positioning platforms embedding cheap off-the-shelf components is a practical choice, it leads not only to differences between the platforms themselves, but also to a high sensitivity to external factors. In this paper, we present a novel lightweight online calibration method composed of two phases, capable of running on miniature robots with limited computational capabilities. Furthermore, by exploiting a Gaussian process regression in its second phase, the proposed calibration approach is able to capture deviations from an assumed underlying physical model. We compare the performance of our approach with the theoretical Cramér-Rao lower bound and test its efficiency on real robots equipped with range and bearing modules.

Details

ICRA Conference 2010 Conference Paper

Towards optimally efficient field estimation with threshold-based pruning in real robotic sensor networks

  • Amanda Prorok
  • Christopher M. Cianci
  • Alcherio Martinoli

The efficiency of distributed sensor networks depends on an optimal trade-off between the usage of resources and data quality. The work in this paper addresses the problem of optimizing this trade-off in a self-configured distributed robotic sensor network, with respect to a user-defined objective function. We investigate a quadtree network topology and implement a fully distributed threshold-based field estimation algorithm. Simulations with field data as well as real robot experiments are performed, validating our distributed control strategy and evaluating the threshold-based formula for real world scenarios. We propose a theoretical analysis that predicts the system's behavior in real world case studies. The experiments and this prediction show very good correspondence, enabling the accurate employment of the objective function, optimizing the trade-off based on user needs.

Details
v2026.09.13