Dirk Ormoneit

UAI Conference 2001 Conference Paper

Lattice Particle Filters

• Dirk Ormoneit
• Christiane Lemieux
• David J. Fleet

A standard approach to approximate inference in state-space models isto apply a particle filter, e.g., the Condensation Algorithm.However, the performance of particle filters often varies significantlydue to their stochastic nature.We present a class of algorithms, called lattice particle filters, thatcircumvent this difficulty by placing the particles deterministicallyaccording to a Quasi-Monte Carlo integration rule.We describe a practical realization of this idea, discuss itstheoretical properties, and its efficiency.Experimental results with a synthetic 2D tracking problem show that thelattice particle filter is equivalent to a conventional particle filterthat has between 10 and 60% more particles, depending ontheir ``sparsity'' in the state-space.We also present results on inferring 3D human motion frommoving light displays.

ICML Conference 2001 Conference Paper

Learning an Agent's Utility Function by Observing Behavior

• Urszula Chajewska
• Daphne Koller
• Dirk Ormoneit

NeurIPS Conference 2001 Conference Paper

Probabilistic Abstraction Hierarchies

• Eran Segal
• Daphne Koller
• Dirk Ormoneit

Many domains are naturally organized in an abstraction hierarchy or taxonomy, where the instances in “nearby” classes in the taxonomy are similar. In this pa- per, we provide a general probabilistic framework for clustering data into a set of classes organized as a taxonomy, where each class is associated with a prob- abilistic model from which the data was generated. The clustering algorithm simultaneously optimizes three things: the assignment of data instances to clus- ters, the models associated with the clusters, and the structure of the abstraction hierarchy. A unique feature of our approach is that it utilizes global optimization algorithms for both of the last two steps, reducing the sensitivity to noise and the propensity to local maxima that are characteristic of algorithms such as hierarchi- cal agglomerative clustering that only take local steps. We provide a theoretical analysis for our algorithm, showing that it converges to a local maximum of the joint likelihood of model and data. We present experimental results on synthetic data, and on real data in the domains of gene expression and text.

UAI Conference 2001 Conference Paper

Robust Combination of Local Controllers

• Carlos Guestrin
• Dirk Ormoneit

Planning problems are hard, motion planning, for example, isPSPACE-hard. Such problems are even more difficult in the presence of uncertainty. Although, Markov Decision Processes (MDPs) provide a formal framework for such problems, finding solutions to high dimensional continuous MDPs is usually difficult, especially when the actions and time measurements are continuous. Fortunately, problem-specific knowledge allows us to design controllers that are good locally, though having no global guarantees. We propose a method of nonparametrically combining local controllers to obtain globally good solutions. We apply this formulation to two types of problems : motion planning (stochastic shortest path) and discounted MDPs. For motion planning, we argue that usual MDP optimality criterion (expected cost) may not be practically relevant. Wepropose an alternative: finding the minimum cost path,subject to the constraint that the robot must reach the goal withhigh probability. For this problem, we prove that a polynomial number of samples is sufficient to obtain a high probability path. For discounted MDPs, we propose a formulation that explicitly deals with model uncertainty, i.e., the problem introduced when transition probabilities are not known exactly. We formulate the problem as a robust linear program which directly incorporates this type of uncertainty.

NeurIPS Conference 2000 Conference Paper

Kernel-Based Reinforcement Learning in Average-Cost Problems: An Application to Optimal Portfolio Choice

• Dirk Ormoneit
• Peter Glynn

Many approaches to reinforcement learning combine neural net(cid: 173) works or other parametric function approximators with a form of temporal-difference learning to estimate the value function of a Markov Decision Process. A significant disadvantage of those pro(cid: 173) cedures is that the resulting learning algorithms are frequently un(cid: 173) stable. In this work, we present a new, kernel-based approach to reinforcement learning which overcomes this difficulty and provably converges to a unique solution. By contrast to existing algorithms, our method can also be shown to be consistent in the sense that its costs converge to the optimal costs asymptotically. Our focus is on learning in an average-cost framework and on a practical ap(cid: 173) plication to the optimal portfolio choice problem.

NeurIPS Conference 2000 Conference Paper

Learning and Tracking Cyclic Human Motion

• Dirk Ormoneit
• Hedvig Sidenbladh
• Michael Black
• Trevor Hastie

We present methods for learning and tracking human motion in video. We estimate a statistical model of typical activities from a large set of 3D periodic human motion data by segmenting these data automatically into "cycles". Then the mean and the princi(cid: 173) pal components of the cycles are computed using a new algorithm that accounts for missing information and enforces smooth tran(cid: 173) sitions between cycles. The learned temporal model provides a prior probability distribution over human motions that can be used in a Bayesian framework for tracking human subjects in complex monocular video sequences and recovering their 3D motion.

NeurIPS Conference 1999 Conference Paper

Optimal Kernel Shapes for Local Linear Regression

• Dirk Ormoneit
• Trevor Hastie

Local linear regression performs very well in many low-dimensional forecasting problems. In high-dimensional spaces, its performance typically decays due to the well-known "curse-of-dimensionality". A possible way to approach this problem is by varying the "shape" of the weighting kernel. In this work we suggest a new, data-driven method to estimating the optimal kernel shape. Experiments us(cid: 173) ing an artificially generated data set and data from the UC Irvine repository show the benefits of kernel shaping.

NeurIPS Conference 1995 Conference Paper

Improved Gaussian Mixture Density Estimates Using Bayesian Penalty Terms and Network Averaging

• Dirk Ormoneit
• Volker Tresp

Volker Tresp