Author name cluster

Budhitama Subagdja

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

1 author row

AAAI Conference 2026 Conference Paper

ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory

Cassandra Hui-Ming Tan
Budhitama Subagdja
Ah-Hwee Tan

Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time. Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information. We introduce Agentic Retrieval with Temporal-Episodic Memory (ARTEM), a hybrid LLM-based agent architecture integrating LLMs with a self-organizing neural network named Spatial-Temporal Episodic Memory (STEM), designed to handle episodic memory tasks. Our approach employs LLMs for event extraction from the inputs to represent temporal, spatial, entitative, and semantic information that may facilitate future retrieval, aside from generating outputs or direct responses. The extracted events can then be encoded vectorially and stored in a fast and stable manner in the episodic memory through an instance-based incremental learning in STEM. STEM supports precise episodes retrieval and helps reduce computational overhead in generating the appropriate responses by LLMs. Evaluation on standardized episodic memory benchmarks across four tasks—partial cue retrieval, epistemic uncertainty detection, recent event identification, and chronological recall—demonstrates superior performance of ARTEM compared to in-context learning, RAG, and fine-tuning in various popular LLMs.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

MEASE: Multi-agent Episodic Action Sequence Explanation

Khaing Phyo Wai
Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Multi-agent reinforcement learning (MARL) achieves remarkable performance in complex coordination tasks, yet interpreting the emergent behaviors of trained agents remains a fundamental challenge. Most current explainability methods focus on individual agent decisions, overlooking the critical interplay of joint strategies and temporal coordination patterns that define successful multiagent policies. We present MEASE (Multi-agent Episodic Action Sequence Explanation), a novel explainable MARL (XMARL) framework that explains trained MARL policies as human-interpretable emergent cooperative joint behaviors. MEASE employs a cognitioninspired episodic memory model to learn spatio-temporal multiagent interaction patterns, coupled with abstraction algorithms that identify significant cooperative agent behaviors. We evaluate MEASE on diverse scenarios in the VMAS and MOSMAC environments, demonstrating its generalizability across various tasks and domains. These explanations, which prescribe “when to do what” for multi-agent systems, serve as executable coordination protocolsthatfaithfullycapturethelearnedbehaviors. Quantitative validation shows that deploying explanations as strategies achieves 93% of the original MARL policy performance. A user study with 31 participants validates the clarity and usefulness of the explanations. These results demonstrate that MEASE effectively extracts explanatory knowledge from complex multi-agent behaviors.

PDF

IJCAI Conference 2025 Conference Paper

L2M2: A Hierarchical Framework Integrating Large Language Model and Multi-agent Reinforcement Learning

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Lin Li
Xin Zhao
Ah-Hwee Tan

Multi-agent reinforcement learning (MARL) has demonstrated remarkable success in collaborative tasks, yet faces significant challenges in scaling to complex scenarios requiring sustained planning and coordination across long horizons. While hierarchical approaches help decompose these tasks, they typically rely on hand-crafted subtasks and domain-specific knowledge, limiting their generalizability. We present L2M2, a novel hierarchical framework that leverages large language models (LLMs) for high-level strategic planning and MARL for low-level execution. L2M2 enables zero-shot planning that supports both end-to-end training and direct integration with pre-trained MARL models. Experiments in the VMAS environment demonstrate that L2M2's LLM-guided MARL achieves superior performance while requiring less than 20% of the training samples compared to baseline methods. In the MOSMAC environment, L2M2 demonstrates strong performance with pre-defined subgoals and maintains substantial effectiveness without subgoals - scenarios where baseline methods consistently fail. Analysis through kernel density estimation reveals L2M2's ability to automatically generate appropriate navigation plans, demonstrating its potential for addressing complex multi-agent coordination tasks.

PDF Details DOI

AAMAS Conference 2025 Conference Paper

MOSMAC: A Multi-agent Reinforcement Learning Benchmark on Sequential Multi-Objective Tasks

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated success on various cooperative multi-agent tasks. However, current benchmarks often fall short of representing realistic scenarios that demand agents to execute sequential tasks over long temporal horizons while balancing multiple objectives. To address this limitation, we introduce multi-objective SMAC (MOS- MAC), a comprehensive MARL benchmark designed to evaluate MARL methods on tasks involving multiple objectives, sequential subtask assignments, and varying temporal horizons. MOSMAC requires agents to tackle a series of interconnected subtasks in Star- Craft II while simultaneously optimizing for multiple objectives, including combat, safety, and navigation. Through rigorous evaluation of nine state-of-the-art MARL algorithms, we demonstrate that MOSMAC presents substantial challenges to existing methods, particularly in long-horizon scenarios. Our analysis establishes MOS- MAC as an essential benchmark for bridging the gap between singleobjective MARL and multi-objective MARL (MOMARL). The codes for MOSMAC are available at: https: //github. com/smu-ncc/mosmac.

PDF

AAMAS Conference 2024 Conference Paper

Benchmarking MARL on Long Horizon Sequential Multi-Objective Tasks

Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

Current MARL benchmarks fall short in simulating realistic scenarios, particularly those involving long action sequences with sequential tasks and multiple conflicting objectives. Addressing this gap, we introduce Multi-Objective SMAC (MOSMAC)1, a novel MARL benchmark tailored to assess MARL methods on tasks with varying time horizons and multiple objectives. Each MOSMAC task contains one or multiple sequential subtasks. Agents are required to simultaneously balance between two objectives — combat and navigation — to successfully complete each subtask. Our evaluation of nine state-of-the-art MARL algorithms reveals that MOSMAC presents substantial challenges to many state-of-the-art MARL methods and effectively fills a critical gap in existing benchmarks for both single-objective and multi-objective MARL research.

PDF

AAMAS Conference 2024 Conference Paper

Explaining Sequences of Actions in Multi-agent Deep Reinforcement Learning Models

Khaing Phyo Wai
Minghong Geng
Shubham Pateria
Budhitama Subagdja
Ah-Hwee Tan

This paper introduces a method to explain MADRL agents’ behaviors by abstracting their actions into high-level strategies. Particularly, a spatio-temporal neural network model is applied to encode the agents’ sequences of actions as memory episodes wherein an aggregating memory retrieval can generalize them into a concise abstract representation of collective strategies. To assess the effectiveness of our method, we applied it to explain the actions of QMIX MADRL agents playing a StarCraft Multi-agent Challenge (SMAC) video game. A user study on the perceived explainability of the extracted strategies indicates that our method can provide comprehensible explanations at various levels of granularity.

PDF

AAMAS Conference 2023 Conference Paper

Towards Explaining Sequences of Actions in Multi-Agent Deep Reinforcement Learning Models

Khaing Phyo Wai
Minghong Geng
Budhitama Subagdja
Shubham Pateria
Ah-Hwee Tan

Although Multi-agent Deep Reinforcement Learning (MADRL) has shown promising results in solving complex real-world problems, the applicability and reliability of MADRL models are often limited by a lack of understanding of their inner workings for explaining the decisions made. To address this issue, this paper proposes a novel method for explaining MADRL by generalizing the sequences of action events performed by agents into high-level abstract strategies using a spatio-temporal neural network model. Specifically, an interval-based memory retrieval procedure is developed to generalize the encoded sequences of action events over time into short sequential patterns. In addition, two abstraction algorithms are introduced, one for abstracting action events across multiple agents and the other for further abstracting the episodes over time into short sequential patterns, which can then be translated into symbolic form for interpretation. We evaluate the proposed method using the StarCraft Multi Agent Challenge (SMAC) benchmark task, which shows that the method is able to derive high-level explanations of MADRL models at various levels of granularity.

PDF

IJCAI Conference 2020 Conference Paper

Who Am I? : Towards Social Self-Awareness for Intelligent Agents

Budhitama Subagdja
Han Yi Tay
Ah-Hwee Tan

Most of today's AI technologies are geared towards mastering specific tasks performance through learning from a huge volume of data. However, less attention has still been given to make the AI understand its own purposes or be responsible socially. In this paper, a new model of agent is presented with the capacity to represent itself as a distinct individual with identity, a mind of its own, unique experiences, and social lives. In this way, the agent can interact with its surroundings and other agents seamlessly and meaningfully. A practical framework for developing an agent architecture with this model of self and self-awareness is proposed allowing self to be ascribed to an existing intelligent agent architecture in general to enable its social ability, interactivity, and co-presence with others. Possible applications are discussed with some exemplifying cases based on an implementation of a conversational agent.

PDF Details DOI

AAMAS Conference 2019 Conference Paper

Beyond Autonomy: The Self and Life of Social Agents: Blue Sky Ideas Track

Budhitama Subagdja
Ah-Hwee Tan

Agents have gained popularity nowadays as virtual assistants and companions of their human users supporting daily activities in many aspects of personal life. Designed to be sociable, an agent engages its user(s) to communicate and even develop friendships. Rather than just as a lifeless toy, it is supposed to be perceived as an individual with its own personality, experiences, and social life. In this paper, we seek to highlight self-hood as another dimension that characterizes an agent. Besides levels of autonomy and reasoning, an agent can be defined based on its capacity to process and reflect on its own self as an individual that possesses identity, embodiment, mind (mental), social relationship with others, and experiences comprising memories about the past and future prospects. We argue that this self-awareness is necessary for a companion agent to engage seamlessly with people as a real actual individual. Some existing implementations and models from preliminary works on agent’s self-awareness illustrate the feasibility and challenges to realize this concept. Beyond assistance and companionship, we also envisage that this model of self is applicable to other types of autonomous application and system involving extensive interaction with people potentially tackling moral and ethical issues.

PDF

AAAI Conference 2017 Conference Paper

Towards a Brain Inspired Model of Self-Awareness for Sociable Agents

Budhitama Subagdja
Ah-Hwee Tan

Self-awareness is a crucial feature for a sociable agent or robot to better interact with humans. In a futuristic scenario, a conversational agent may occasionally be asked for its own opinion or suggestion based on its own thought, feelings, or experiences as if it is an individual with identity, personality, and social life. In moving towards that direction, in this paper, a brain inspired model of self-awareness is presented that allows an agent to learn to attend to different aspects of self as an individual with identity, physical embodiment, mental states, experiences, and reﬂections on how others may think about oneself. The model is built and realized on a NAO humanoid robotic platform to investigate the role of this capacity of self-awareness on the robot’s learning and interactivity.

PDF Details

AAMAS Conference 2012 Conference Paper

Memory Formation, Consolidation, and Forgetting in Learning Agents

Budhitama Subagdja
Wenwen Wang
Ah-Hwee Tan
Yuan-Sin Tan
Loo-Nin Teow

Memory enables past experiences to be remembered and acquired as useful knowledge to support decision making, especially when perception and computational resources are limited. This paper presents a neuropsychological-inspired dual memory model for agents, consisting of an episodic memory that records the agent's experience in real time and a semantic memory that captures factual knowledge through a parallel consolidation process. In addition, the model incorporates a natural forgetting mechanism that prevents memory overloading by removing transient memory traces. Our experimental study based on a real-time first-person-shooter video game has indicated that the memory consolidation and forgetting processes are not only able to extract valuable knowledge and regulate the memory capacity, but they can mutually improve the effectiveness of learning the knowledge for the given task in hand. Interestingly, a moderate level of forgetting may even improve the task performance rather than disadvantaging it. We suggest that the interplay between rapid memory formation, consolidation, and forgetting processes points to a practical and effective approach for learning agents to acquire and maintain useful knowledge from experiences in a scalable manner.

PDF

AAMAS Conference 2012 Conference Paper

Virtual Characters in Agent-Augmented Co-Space

Yi-Lin Kang
Budhitama Subagdja
Ah-Hwee Tan
Yew-Soon Ong
Chunyan Miao

Co-Space refers to interactive virtual environment modelled after the real world in terms of look-and-feel, functionalities and services. We have developed a 3D virtual world named Nanyang Technological University (NTU) Co-Space populated with virtual human characters. In order to create realistic virtual humans, we have designed a brain-inspired agent architecture with the properties of goal-directed autonomy, natural interactivity and human-like personification. The demo will show how the virtual characters may enhance the interactivity and playability of the virtual worlds.

PDF

AAMAS Conference 2009 Conference Paper

A Self-Organizing Neural Network Architecture for Intentional Planning Agents

Budhitama Subagdja
Ah-Hwee Tan

This paper presents a model of neural network embodiment of intentions and planning mechanisms for autonomous agents. The model bridges the dichotomy of symbolic and non-symbolic representation in developing agents. Some novel techniques are introduced that enables the neural network to process and manipulate sequential and hierarchical structures of information. It is suggested that by incorporating intentional agent model which relies on explicit symbolic description with self-organizing neural networks that are good at learning and recognizing patterns, the best from both sides can be exploited. This paper demonstrates that plans can be represented as weighted connections and reasoning processes can be accommodated through multidirectional activations accross diﬀerent modalities of patterns. The network seamlessly interleaves planning and learning processes towards achieving the goal. Case studies and experiments shows that the model can be used to execute, plan, and capture plans as recipes through experiences.

PDF

JAAMAS Journal 2008 Journal Article

Intentional learning agent architecture

Budhitama Subagdja
Liz Sonenberg
Iyad Rahwan

Abstract Dealing with changing situations is a major issue in building agent systems. When the time is limited, knowledge is unreliable, and resources are scarce, the issue becomes more challenging. The BDI (Belief-Desire-Intention) agent architecture provides a model for building agents that addresses that issue. The model can be used to build intentional agents that are able to reason based on explicit mental attitudes, while behaving reactively in changing circumstances. However, despite the reactive and deliberative features, a classical BDI agent is not capable of learning. Plans as recipes that guide the activities of the agent are assumed to be static. In this paper, an architecture for an intentional learning agent is presented. The architecture is an extension of the BDI architecture in which the learning process is explicitly described as plans. Learning plans are meta-level plans which allow the agent to introspectively monitor its mental states and update other plans at run time. In order to acquire the intricate structure of a plan, a process pattern called manipulative abduction is encoded as a learning plan. This work advances the state of the art by combining the strengths of learning and BDI agent frameworks in a rich language for describing deliberation processes and reactive execution. It enables domain experts to specify learning processes and strategies explicitly, while allowing the agent to benefit from procedural domain knowledge expressed in plans.

Details DOI