Geometric Variance Reduction in Markov Chains: Application to Value Function and Gradient Estimation

We study a variance reduction technique for Monte Carlo estimation of functionals in Markov chains. The method is based on designing sequential control variates using successive approximations of the function of interest V. Regular Monte Carlo estimates have a variance of O(1/N), where N is the number of sample trajectories of the Markov chain. Here, we obtain a geometric variance reduction O(ρ N ) (with ρ [abs] [ pdf ][ bib ] &copy JMLR 2006. ( edit, beta )

Authors

• Rémi Munos

Keywords

No keywords are indexed for this paper.