Arrow Research search

Author name cluster

Piotr Gmytrasiewicz

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers
1 author row

Possible papers

11

JAIR Journal 2020 Journal Article

How to Do Things with Words: A Bayesian Approach

  • Piotr Gmytrasiewicz

Communication changes the beliefs of the listener and of the speaker. The value of a communicative act stems from the valuable belief states which result from this act. To model this we build on the Interactive POMDP (IPOMDP) framework, which extends POMDPs to allow agents to model others in multi-agent settings, and we include communication that can take place between the agents to formulate Communicative IPOMDPs (CIPOMDPs). We treat communication as a type of action and therefore, decisions regarding communicative acts are based on decision-theoretic planning using the Bellman optimality principle and value iteration, just as they are for all other rational actions. As in any form of planning, the results of actions need to be precisely specified. We use the Bayes’ theorem to derive how agents update their beliefs in CIPOMDPs; updates are due to agents’ actions, observations, messages they send to other agents, and messages they receive from others. The Bayesian decision-theoretic approach frees us from the commonly made assumption of cooperative discourse – we consider agents which are free to be dishonest while communicating and are guided only by their selfish rationality. We use a simple Tiger game to illustrate the belief update, and to show that the ability to rationally communicate allows agents to improve efficiency of their interactions.

PDF Details DOI

AIJ Journal 2020 Journal Article

Recursively modeling other agents for decision making: A research perspective

  • Prashant Doshi
  • Piotr Gmytrasiewicz
  • Edmund Durfee

Individuals exhibit theory of mind, attributing beliefs, intent, and mental states to others as explanations of observed actions. Dennett's intentional stance offers an analogous abstraction for computational agents seeking to understand, explain, or predict others' behaviors. These recognized theories provide a formal basis to ongoing investigations of recursive modeling. We review and situate various frameworks for recursive modeling that have been studied in game- and decision- theories, and have yielded methods useful to AI researchers. Sustained attention given to these frameworks has produced new analyses and methods with an aim toward making recursive modeling practicable. Indeed, we also review some emerging uses and the insights these yielded, which are indicative of pragmatic progress in this area. The significance of these frameworks is that higher-order reasoning is critical to correctly recognizing others' intent or outthinking opponents. Such reasoning has been utilized in academic, business, military, security, and other contexts both to train and inform decision-making agents in organizational and strategic contexts, and also to more realistically predict and best respond to other agents' intent.

Details DOI

AAAI Conference 2019 Conference Paper

IPOMDP-Net: A Deep Neural Network for Partially Observable Multi-Agent Planning Using Interactive POMDPs

  • Yanlin Han
  • Piotr Gmytrasiewicz

This paper introduces the IPOMDP-net, a neural network architecture for multi-agent planning under partial observability. It embeds an interactive partially observable Markov decision process (I-POMDP) model and a QMDP planning algorithm that solves the model in a neural network architecture. The IPOMDP-net is fully differentiable and allows for end-to-end training. In the learning phase, we train an IPOMDP-net on various fixed and randomly generated environments in a reinforcement learning setting, assuming observable reinforcements and unknown (randomly initialized) model functions. In the planning phase, we test the trained network on new, unseen variants of the environments under the planning setting, using the trained model to plan without reinforcements. Empirical results show that our model-based IPOMDP-net outperforms the other state-of-the-art modelfree network and generalizes better to larger, unseen environments. Our approach provides a general neural computing architecture for multi-agent planning using I-POMDPs. It suggests that, in a multi-agent setting, having a model of other agents benefits our decision-making, resulting in a policy of higher quality and better generalizability.

PDF Details

AAMAS Conference 2019 Conference Paper

Optimal Sequential Planning for Communicative Actions: A Bayesian Approach

  • Piotr Gmytrasiewicz
  • Sarit Adhikari

We build on the Interactive POMDP (IPOMDP) framework, which extends POMDPs to multi-agent settings, and include communication which can take place between the agents. While IPOMDPs endow the agents with models of their environments and models of other agents, we supplement IPOMDPs with communicative acts available to the agents to formulate Communicative IPOMDPs (CIPOMDPs). We treat communication as a type of action; hence decisions regarding communicative acts should be based on decisiontheoretic planning using Bellman optimality principle, just as they are for all other actions. As in any form of planning, the results of actions need to be precisely specified. We use Bayes update to derive how agents update their beliefs in CIPOMDPs; updates are due to their actions, observations, messages they send to other agents, and messages they receive from others. Without communication CIPOMDPs reduce to IPOMDPs. Without other agents they all become classical POMDPs.

PDF

NeurIPS Conference 2018 Conference Paper

Learning Others' Intentional Models in Multi-Agent Settings Using Interactive POMDPs

  • Yanlin Han
  • Piotr Gmytrasiewicz

Interactive partially observable Markov decision processes (I-POMDPs) provide a principled framework for planning and acting in a partially observable, stochastic and multi-agent environment. It extends POMDPs to multi-agent settings by including models of other agents in the state space and forming a hierarchical belief structure. In order to predict other agents' actions using I-POMDPs, we propose an approach that effectively uses Bayesian inference and sequential Monte Carlo sampling to learn others' intentional models which ascribe to them beliefs, preferences and rationality in action selection. Empirical results show that our algorithm accurately learns models of the other agent and has superior performance than methods that use subintentional models. Our approach serves as a generalized Bayesian learning algorithm that learns other agents' beliefs, strategy levels, and transition, observation and reward functions. It also effectively mitigates the belief space complexity due to the nested belief hierarchy.

PDF Details

JAAMAS Journal 2017 Journal Article

Interactive POMDPs with finite-state models of other agents

  • Alessandro Panella
  • Piotr Gmytrasiewicz

Abstract We consider an autonomous agent facing a stochastic, partially observable, multiagent environment. In order to compute an optimal plan, the agent must accurately predict the actions of the other agents, since they influence the state of the environment and ultimately the agent’s utility. To do so, we propose a special case of interactive partially observable Markov decision process, in which the agent does not explicitly model the other agents’ beliefs and preferences, and instead represents them as stochastic processes implemented by probabilistic deterministic finite state controllers (PDFCs). The agent maintains a probability distribution over the PDFC models of the other agents, and updates this belief using Bayesian inference. Since the number of nodes of these PDFCs is unknown and unbounded, the agent places a Bayesian nonparametric prior distribution over the infinitely dimensional set of PDFCs. This allows the size of the learned models to adapt to the complexity of the observed behavior. Deriving the posterior distribution is in this case too complex to be amenable to analytical computation; therefore, we provide a Markov chain Monte Carlo algorithm that approximates the posterior beliefs over the other agents’ PDFCs, given a sequence of (possibly imperfect) observations about their behavior. Experimental results show that the learned models converge behaviorally to the true ones. We consider two settings, one in which the agent first learns, then interacts with other agents, and one in which learning and planning are interleaved. We show that the agent’s performance increases as a result of learning in both situations. Moreover, we analyze the dynamics that ensue when two agents are simultaneously learning about each other while interacting, showing in an example environment that coordination emerges naturally from our approach. Furthermore, we demonstrate how an agent can exploit the learned models to perform indirect inference over the state of the environment via the modeled agent’s actions.

Details DOI

AAAI Conference 2016 Conference Paper

Bayesian Learning of Other Agents’ Finite Controllers for Interactive POMDPs

  • Alessandro Panella
  • Piotr Gmytrasiewicz

We consider an autonomous agent operating in a stochastic, partially-observable, multiagent environment, that explicitly models the other agents as probabilistic deterministic finitestate controllers (PDFCs) in order to predict their actions. We assume that such models are not given to the agent, but instead must be learned from (possibly imperfect) observations of the other agents’ behavior. The agent maintains a belief over the other agents’ models, that is updated via Bayesian inference. To represent this belief we place a flexible stickbreaking distribution over PDFCs, that allows the posterior to concentrate around controllers whose size is not bounded and scales with the complexity of the observed data. Since this Bayesian inference task is not analytically tractable, we devise a Markov chain Monte Carlo algorithm to approximate the posterior distribution. The agent then embeds the result of this inference into its own decision making process using the interactive POMDP framework. We show that our learning algorithm can learn agent models that are behaviorally accurate for problems of varying complexity, and that the agent’s performance increases as a result.

PDF Details

AAMAS Conference 2011 Conference Paper

Modeling Bounded Rationality of Agents During Interactions

  • Qing Guo
  • Piotr Gmytrasiewicz

In this paper, we propose that bounded rationality of another agent be modeled as errors the agent is making while deciding on its action. We are motivated by the work on quantal response equilibria in behavioral game theory which uses Nash equilibria as the solution concept. In contrast, we use decision-theoretic maximization of expected utility. Quantal response assumes that a decision maker is approximately rational, i. e. , is maximizing its expected utility but with an error rate characterized by a single error parameter. Another agent's error rate may be unknown and needs to be estimated during an interaction. We show that this error rate can be estimated using Bayesian update of a suitable conjugate prior, and that it has a sufficient statistic of fixed dimension under strong simplifying assumptions. However, if the simplifying assumptions are relaxed, the quantal response does not admit a finite dimensional sufficient statistic, and a more complex update is needed.

PDF
v2026.09.13