Author name cluster

Konstantinos Blekas

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

JMLR Journal 2014 Journal Article

Cover Tree Bayesian Reinforcement Learning

Nikolaos Tziortziotis
Christos Dimitrakakis
Konstantinos Blekas

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tree method, which remains efficient in high dimensional spaces. We combine the model with Thompson sampling and approximate dynamic programming to obtain effective exploration policies in unknown environments. The flexibility and computational simplicity of the model render it suitable for many reinforcement learning problems in continuous state spaces. We demonstrate this in an experimental comparison with a Gaussian process model, a linear model and simple least squares policy iteration. [abs] [ pdf ][ bib ] &copy JMLR 2014. ( edit, beta )

PDF Details

IJCAI Conference 2013 Conference Paper

Linear Bayesian Reinforcement Learning

Nikolaos Tziortziotis
Christos Dimitrakakis
Konstantinos Blekas

This paper proposes a simple linear Bayesian approach to reinforcement learning. We show that with an appropriate basis, a Bayesian linear Gaussian model is sufficient for accurately estimating the system dynamics, and in particular when we allow for correlated noise. Policies are estimated by first sampling a transition model from the current posterior, and then performing approximate dynamic programming on the sampled model. This form of approximate Thompson sampling results in good exploration in unknown environments. The approach can also be seen as a Bayesian generalisation of least-squares policy iteration, where the empirical transition matrix is replaced with a sample from the posterior.

PDF Details DOI

AAAI Conference 2011 Conference Paper

A Bayesian Reinforcement Learning framework Using Relevant Vector Machines

Nikolaos Tziortziotis
Konstantinos Blekas

In this work we present an advanced Bayesian formulation to the task of control learning that employs the Relevance Vector Machines (RVM) generative model for value function evaluation. The key aspect of the proposed method is the design of the discount return as a generalized linear model that constitutes a wellknown probabilistic approach. This allows to augment the model with advantageous sparse priors provided by the RVM’s regression framework. We have also taken into account the signiﬁcant issue of selecting the proper parameters of the kernel design matrix. Experiments have shown that our method produces improved performance in both simulated and real test environments.

PDF Details

EWRL Workshop 2011 Conference Paper

Value Function Approximation through Sparse Bayesian Modeling

Nikolaos Tziortziotis
Konstantinos Blekas

Abstract In this study we present a sparse Bayesian framework for value function approximation. The proposed method is based on the on-line construction of a dictionary of states which are collected during the exploration of the environment by the agent. A linear regression model is established for the observed partial discounted return of such dictionary states, where we employ the Relevance Vector Machine (RVM) and exploit its enhanced modeling capability due to the embedded sparsity properties. In order to speed-up the optimization procedure and allow dealing with large-scale problems, an incremental strategy is adopted. A number of experiments have been conducted on both simulated and real environments, where we took promising results in comparison with another Bayesian approach that uses Gaussian processes.

Details