Brenda Ng

AAAI Conference 2012 Conference Paper

Bayes-Adaptive Interactive POMDPs

• Brenda Ng
• Kofi Boakye
• Carol Meyers
• Andrew Wang

We introduce the Bayes-Adaptive Interactive Partially Observable Markov Decision Process (BA-IPOMDP), the first multiagent decision model that explicitly incorporates model learning. As in I-POMDPs, the BA-IPOMDP agent maintains beliefs over interactive states, which include the physical states as well as the other agents’ models. The BA-IPOMDP assumes that the state transition and observation probabilities are unknown, and augments the interactive states to include these parameters. Beliefs are maintained over this augmented interactive state space. This (necessary) state expansion exacerbates the curse of dimensionality, especially since each I-POMDP belief update is already a recursive procedure (because an agent invokes belief updates from other agents’ perspectives as part of its own belief update, in order to anticipate other agents’ actions). We extend the interactive particle filter to perform approximate belief update on BA-IPOMDPs. We present our findings on the multiagent Tiger problem.

IJCAI Conference 2007 Conference Paper

• Avi Pfeffer
• Subrata Das
• David Lawless
• Brenda Ng

Many dynamic systems involve a number of entities that are largely independent of each other but interact with each other via a subset of state variables. We present global/local dynamic models (GLDMs) to capture these kinds of systems. In a GLDM, the state of an entity is decomposed into a globally influenced state that depends on other entities, and a locally influenced state that depends only on the entity itself. We present an inference algorithm for GLDMs called global/local particle filtering, that introduces the principle of reasoning globally about global dynamics and locally about local dynamics. We have applied GLDMs to an asymmetric urban warfare environment, in which enemy units form teams to attack important targets, and the task is to detect such teams as they form. Experimental results for this application show that global/local particle filtering outperforms ordinary particle filtering and factored particle filtering.

IJCAI Conference 2005 Conference Paper

Continuous Time Particle Filtering

• Brenda Ng
• Avi Pfeffer
• Richard

We present the continuous-time particle filter (CTPF) – an extension of the discrete-time particle filter for monitoring continuous-time dynamic systems. Our methods apply to hybrid systems containing both discrete and continuous variables. The dynamics of the discrete state system are governed by a Markov jump process. Observations of the discrete process are intermittent and irregular. Whenever the discrete process is observed, CTPF samples a trajectory of the underlying Markov jump process. This trajectory is then used to estimate the continuous variables using the system dynamics determined by the discrete state in the trajectory. We use the unscented Kalman-Bucy filter to handle nonlinearities and continuous time. We present results showing that CTPF is more stable in its performance than discrete-time particle filtering, even when the discrete-time algorithm is allowed to update many more times than CTPF. We also present a method for online learning of the Markov jump process model that governs the discrete states.

UAI Conference 2002 Conference Paper

Factored Particles for Scalable Monitoring

• Brenda Ng
• Leonid Peshkin
• Avi Pfeffer

Exact monitoring in dynamic Bayesian networks is intractable, so approximate algorithms are necessary. This paper presents a new family of approximate monitoring algorithms that combine the best qualities of the particle filtering and Boyen-Koller methods. Our algorithms maintain an approximate representation the belief state in the form of sets of factored particles, that correspond to samples of clusters of state variables. Empirical results show that our algorithms outperform both ordinary particle filtering and the Boyen-Koller algorithm on large systems.