Author name cluster

Shubham Pateria

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

AAMAS Conference 2026 Conference Paper

MEASE: Multi-agent Episodic Action Sequence Explanation

Khaing Phyo Wai
Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Multi-agent reinforcement learning (MARL) achieves remarkable performance in complex coordination tasks, yet interpreting the emergent behaviors of trained agents remains a fundamental challenge. Most current explainability methods focus on individual agent decisions, overlooking the critical interplay of joint strategies and temporal coordination patterns that define successful multiagent policies. We present MEASE (Multi-agent Episodic Action Sequence Explanation), a novel explainable MARL (XMARL) framework that explains trained MARL policies as human-interpretable emergent cooperative joint behaviors. MEASE employs a cognitioninspired episodic memory model to learn spatio-temporal multiagent interaction patterns, coupled with abstraction algorithms that identify significant cooperative agent behaviors. We evaluate MEASE on diverse scenarios in the VMAS and MOSMAC environments, demonstrating its generalizability across various tasks and domains. These explanations, which prescribe “when to do what” for multi-agent systems, serve as executable coordination protocolsthatfaithfullycapturethelearnedbehaviors. Quantitative validation shows that deploying explanations as strategies achieves 93% of the original MARL policy performance. A user study with 31 participants validates the clarity and usefulness of the explanations. These results demonstrate that MEASE effectively extracts explanatory knowledge from complex multi-agent behaviors.

PDF

IJCAI Conference 2025 Conference Paper

L2M2: A Hierarchical Framework Integrating Large Language Model and Multi-agent Reinforcement Learning

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Lin Li
Xin Zhao
Ah-Hwee Tan

Multi-agent reinforcement learning (MARL) has demonstrated remarkable success in collaborative tasks, yet faces significant challenges in scaling to complex scenarios requiring sustained planning and coordination across long horizons. While hierarchical approaches help decompose these tasks, they typically rely on hand-crafted subtasks and domain-specific knowledge, limiting their generalizability. We present L2M2, a novel hierarchical framework that leverages large language models (LLMs) for high-level strategic planning and MARL for low-level execution. L2M2 enables zero-shot planning that supports both end-to-end training and direct integration with pre-trained MARL models. Experiments in the VMAS environment demonstrate that L2M2's LLM-guided MARL achieves superior performance while requiring less than 20% of the training samples compared to baseline methods. In the MOSMAC environment, L2M2 demonstrates strong performance with pre-defined subgoals and maintains substantial effectiveness without subgoals - scenarios where baseline methods consistently fail. Analysis through kernel density estimation reveals L2M2's ability to automatically generate appropriate navigation plans, demonstrating its potential for addressing complex multi-agent coordination tasks.

PDF Details DOI

AAMAS Conference 2025 Conference Paper

MOSMAC: A Multi-agent Reinforcement Learning Benchmark on Sequential Multi-Objective Tasks

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated success on various cooperative multi-agent tasks. However, current benchmarks often fall short of representing realistic scenarios that demand agents to execute sequential tasks over long temporal horizons while balancing multiple objectives. To address this limitation, we introduce multi-objective SMAC (MOS- MAC), a comprehensive MARL benchmark designed to evaluate MARL methods on tasks involving multiple objectives, sequential subtask assignments, and varying temporal horizons. MOSMAC requires agents to tackle a series of interconnected subtasks in Star- Craft II while simultaneously optimizing for multiple objectives, including combat, safety, and navigation. Through rigorous evaluation of nine state-of-the-art MARL algorithms, we demonstrate that MOSMAC presents substantial challenges to existing methods, particularly in long-horizon scenarios. Our analysis establishes MOS- MAC as an essential benchmark for bridging the gap between singleobjective MARL and multi-objective MARL (MOMARL). The codes for MOSMAC are available at: https: //github. com/smu-ncc/mosmac.

PDF

AAMAS Conference 2024 Conference Paper

Benchmarking MARL on Long Horizon Sequential Multi-Objective Tasks

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Current MARL benchmarks fall short in simulating realistic scenarios, particularly those involving long action sequences with sequential tasks and multiple conflicting objectives. Addressing this gap, we introduce Multi-Objective SMAC (MOSMAC)1, a novel MARL benchmark tailored to assess MARL methods on tasks with varying time horizons and multiple objectives. Each MOSMAC task contains one or multiple sequential subtasks. Agents are required to simultaneously balance between two objectives — combat and navigation — to successfully complete each subtask. Our evaluation of nine state-of-the-art MARL algorithms reveals that MOSMAC presents substantial challenges to many state-of-the-art MARL methods and effectively fills a critical gap in existing benchmarks for both single-objective and multi-objective MARL research.

PDF

AAMAS Conference 2024 Conference Paper

Explaining Sequences of Actions in Multi-agent Deep Reinforcement Learning Models

Khaing Phyo Wai
Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

This paper introduces a method to explain MADRL agents’ behaviors by abstracting their actions into high-level strategies. Particularly, a spatio-temporal neural network model is applied to encode the agents’ sequences of actions as memory episodes wherein an aggregating memory retrieval can generalize them into a concise abstract representation of collective strategies. To assess the effectiveness of our method, we applied it to explain the actions of QMIX MADRL agents playing a StarCraft Multi-agent Challenge (SMAC) video game. A user study on the perceived explainability of the extracted strategies indicates that our method can provide comprehensible explanations at various levels of granularity.

PDF

AAMAS Conference 2023 Conference Paper

Towards Explaining Sequences of Actions in Multi-Agent Deep Reinforcement Learning Models

Khaing Phyo Wai
Minghong Geng
Budhitama Subagdja
Shubham Pateria
Ah-Hwee Tan

Although Multi-agent Deep Reinforcement Learning (MADRL) has shown promising results in solving complex real-world problems, the applicability and reliability of MADRL models are often limited by a lack of understanding of their inner workings for explaining the decisions made. To address this issue, this paper proposes a novel method for explaining MADRL by generalizing the sequences of action events performed by agents into high-level abstract strategies using a spatio-temporal neural network model. Specifically, an interval-based memory retrieval procedure is developed to generalize the encoded sequences of action events over time into short sequential patterns. In addition, two abstraction algorithms are introduced, one for abstracting action events across multiple agents and the other for further abstracting the episodes over time into short sequential patterns, which can then be translated into symbolic form for interpretation. We evaluate the proposed method using the StarCraft Multi Agent Challenge (SMAC) benchmark task, which shows that the method is able to derive high-level explanations of MADRL models at various levels of granularity.

PDF