Learning Predictive State Representations for planning

Predictive State Representations (PSRs) allow modeling of dynamical systems directly in observables and without relying on latent variable representations. A problem that arises from learning PSRs is that it is often hard to attribute semantic meaning to the learned representation. This makes generalization and planning in PSRs challenging. In this paper, we extend PSRs and introduce the notion of PSRs that include prior information (P-PSRs) to learn representations which are suitable for planning and interpretation. By learning a low-dimensional embedding of test features we map belief points of similar semantic to the same region of a subspace. This facilitates better generalization for planning and semantical interpretation of the learned representation. In specific, we show how to overcome the training sample bias and introduce feature selection such that the resulting representation emphasizes observables related to the planning task. We show that our P-PSRs result in qualitatively meaningful representations and present quantitative results that indicate improved suitability for planning.

Authors

• Johannes A. Stork
• Carl Henrik Ek
• Danica Kragic

Keywords

• Planning
• History
• Robots
• Semantics
• Predictive models
• Uncertainty
• Representation Learning
• Task Planning
• Low-dimensional Embedding
• Training Data
• Singular Value Decomposition
• Latent Space
• Structure Of Space
• Projection Matrix
• Test Characteristics
• Spectral Method
• Pseudo-inverse
• Policy Learning
• Historical Features
• Sequence Of Observations
• Spectral Model
• Value Iteration
• Sufficient Statistics
• Robot Navigation
• Observable Quantities
• Continuous Domain
• Spectral Approach
• Temporal Coherence
• RGB Space
• Sequence Of Actions
• Optimization Problem
• Learning Algorithms