Composable Learning with Sparse Kernel Representations

We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function (NAF). This representation of the policy enables efficiently composing multiple learned models without additional training samples or interaction with the environment. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment. We apply the composition operation to various policy combinations and test them to show that the composed policies retain the performance of their components. We also transfer the composed policy directly to a physical platform operating in an arena with obstacles in order to demonstrate a degree of generalization.

Authors

• Ekaterina I. Tolstaya
• Ethan Stump
• Alec Koppel
• Alejandro Ribeiro

Keywords

• Kernel
• Stochastic processes
• Hilbert space
• Data models
• Training
• Complexity theory
• Robots
• Sparse Kernel
• Laser Scanning
• Reproducing Kernel Hilbert Space
• 2D Environment
• Collision
• Time Step
• Wireless
• State Space
• Fixed Point
• Angular Velocity
• Kernel Function
• Continuous Action
• Kernel Density
• Test Environment
• Optimal Policy
• Training Step
• Sparse Representation
• Reward Function
• Markov Decision Process
• Action-value Function
• Policy Learning
• Multiple Policy
• Directional Derivative
• Laser Ranging
• Stochastic Policy