Abstracts Category : Other

Learning distributions with Particle Mirror Descent

by Ailing Zhang

As an effective and provable primal method to estimate posterior distribution, Particle Mirror Descent is appealing for its simplicity and flexibility. In this thesis we explore the applications of Particle Mirror Descent in both supervised and unsupervised learning. In the general classification problem with a parametric discriminative function, we seek a posterior distribution over parameters that maximizes classification accuracy. Existing algorithms usually solve the dual problem and the number of variables to optimize depends on the number of examples in the dataset. Therefore such methods suffer from the poor scalability in large datasets. We propose Constrained Particle Mirror Descent to effectively estimate posterior distribution in primal space even with expectation constraint. By marrying Bayesian probabilistic inference and deep neural networks, deep generative networks have shown remarkable success in various kinds of generative tasks. However, such models usually make an assumption that posterior distribution can be simply characterized as a Gaussian distribution, which is not always true since real data like images and audios yield complex structures in latent space. Motivated by the recent wide application of multi-modal posterior, we introduce a variant of Variational Auto-encoder model that uses a mixture of customized kernels as posterior distribution in latent space. Our deep generative model produces visually plausible images as well as good clustering performance using latent representations.Advisors/Committee Members: Koyejo, Oluwasanmi (advisor), He, Niao (advisor).