Student-t Variational Autoencoder for Robust Density Estimation

We propose a robust multivariate density estimator based on the variational autoencoder (VAE). The VAE is a powerful deep generative model, and used for multivariate density estimation. With the original VAE, the distribution of observed continuous variables is assumed to be a Gaussian, where its mean and variance are modeled by deep neural networks taking latent variables as their inputs. This distribution is called the decoder. However, the training of VAE often becomes unstable. One reason is that the decoder of VAE is sensitive to the error between the data point and its estimated mean when its estimated variance is almost zero. We solve this instability problem by making the decoder robust to the error using a Bayesian approach to the variance estimation: we set a prior for the variance of the Gaussian decoder, and marginalize it out analytically, which leads to proposing the Student-t VAE. Numerical experiments with various datasets show that training of the Student-t VAE is robust, and the Student-t VAE achieves high density estimation performance.

Authors

• Hiroshi Takahashi
• Tomoharu Iwata NTT Communication Science Laboratories
• Yuki Yamanaka
• Masanori Yamada
• Satoshi Yagi

Keywords

• Machine Learning: Deep Learning
• Machine Learning: Machine Learning
• Machine Learning: Neural Networks
• Machine Learning: Probabilistic Machine Learning
• Machine Learning: Unsupervised Learning