Siddharth Nayak

AAMAS Conference 2025 Conference Paper

Asynchronous Cooperative Multi-Agent Reinforcement Learning with Limited Communication

• Sydney Dolan
• Siddharth Nayak
• Jasmine Jerry Aloor
• Hamsa Balakrishnan

We consider the problem setting in which multiple autonomous agents must cooperatively navigate and perform tasks in an unknown, communication-constrained environment. Traditional multiagent reinforcement learning (MARL) approaches assume synchronous communications and perform poorly in such environments. We propose AsynCoMARL, an asynchronous MARL approach that uses graph transformers to learn communication protocols from dynamic graphs. AsynCoMARL can accommodate infrequent and asynchronous communications between agents, with edges of the graph only forming when agents communicate with each other. We show that AsynCoMARL achieves similar success and collision rates as leading baselines, despite 26% fewer messages being passed between agents.

NeurIPS Conference 2024 Conference Paper

Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments

• Siddharth Nayak
• Adelmo M. Orozco
• Marina T. Have
• Vittal Thirumalai
• Jackson Zhang
• Darren Chen
• Aditya Kapoor
• Eric Robinson

The ability of Language Models (LMs) to understand natural language makes them a powerful tool for parsing human instructions into task plans for autonomous robots. Unlike traditional planning methods that rely on domain-specific knowledge and handcrafted rules, LMs generalize from diverse data and adapt to various tasks with minimal tuning, acting as a compressed knowledge base. However, LMs in their standard form face challenges with long-horizon tasks, particularly in partially observable multi-agent settings. We propose an LM-based Long-Horizon Planner for Multi-Agent Robotics (LLaMAR), a cognitive architecture for planning that achieves state-of-the-art results in long-horizon tasks within partially observable environments. LLaMAR employs a plan-act-correct-verify framework, allowing self-correction from action execution feedback without relying on oracles or simulators. Additionally, we present MAP-THOR, a comprehensive test suite encompassing household tasks of varying complexity within the AI2-THOR environment. Experiments show that LLaMAR achieves a 30\% higher success rate than other state-of-the-art LM-based multi-agent planners in MAP-THOR and Search & Rescue tasks. Code can be found at https: //github. com/nsidn98/LLaMAR

YNICL Journal 2023 Journal Article

Distinct and joint effects of low and high levels of Aβ and tau deposition on cortical thickness

• Seyed Hani Hojjati
• Tracy A. Butler
• Gloria C. Chiang
• Christian Habeck
• Arindam RoyChoudhury
• Farnia Feiz
• Jacob Shteingart
• Siddharth Nayak

ICML Conference 2023 Conference Paper

Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation

• Siddharth Nayak
• Kenneth Choi
• Wenqi Ding
• Sydney Dolan
• Karthik Gopalakrishnan 0002
• Hamsa Balakrishnan

We consider the problem of multi-agent navigation and collision avoidance when observations are limited to the local neighborhood of each agent. We propose InforMARL, a novel architecture for multi-agent reinforcement learning (MARL) which uses local information intelligently to compute paths for all the agents in a decentralized manner. Specifically, InforMARL aggregates information about the local neighborhood of agents for both the actor and the critic using a graph neural network and can be used in conjunction with any standard MARL algorithm. We show that (1) in training, InforMARL has better sample efficiency and performance than baseline approaches, despite using less information, and (2) in testing, it scales well to environments with arbitrary numbers of agents and obstacles. We illustrate these results using four task environments, including one with predetermined goals for each agent, and one in which the agents collectively try to cover all goals.

AAAI Conference 2022 Conference Paper

NICE: Robust Scheduling through Reinforcement Learning-Guided Integer Programming

• Luke Kenworthy
• Siddharth Nayak
• Christopher Chin
• Hamsa Balakrishnan

Integer programs provide a powerful abstraction for representing a wide range of real-world scheduling problems. Despite their ability to model general scheduling problems, solving large-scale integer programs (IP) remains a computational challenge in practice. The incorporation of more complex objectives such as robustness to disruptions further exacerbates the computational challenge. We present NICE (Neural network IP Coefficient Extraction), a novel technique that combines reinforcement learning and integer programming to tackle the problem of robust scheduling. More specifically, NICE uses reinforcement learning to approximately represent complex objectives in an integer programming formulation. We use NICE to determine assignments of pilots to a flight crew schedule so as to reduce the impact of disruptions. We compare NICE with (1) a baseline integer programming formulation that produces a feasible crew schedule, and (2) a robust integer programming formulation that explicitly tries to minimize the impact of disruptions. Our experiments show that, across a variety of scenarios, NICE produces schedules resulting in 33% to 48% fewer disruptions than the baseline formulation. Moreover, in more severely constrained scheduling scenarios in which the robust integer program fails to produce a schedule within 90 minutes, NICE is able to build robust schedules in less than 2 seconds on average.