MaxEntropy Pursuit Variational Inference

I presented our (Evgenii Egorov, Kirill Neklydov, Ruslan Kostoev and Evgeny Burnaev) work on greedy semi-parametric Variational Inference. We propose a version of the Frank-Wolfe algorithm with entropy regularization in the density space. There are slides and full-text on arXiv, as well as published version.

Abstract of the paper:

One of the core problems in variational inference is a choice of approximate posterior distribution. It is crucial to trade-off between efficient inference with simple families as mean-field models and accuracy of inference. We propose a variant of a greedy approximation of the posterior distribution with tractable base learners. Using Max-Entropy approach, we obtain a well-defined optimization problem. We demonstrate the ability of the method to capture complex multimodal posterior via continual learning setting for neural networks.