Summary of Stochastic Variance-reduced Gaussian Variational Inference on the Bures-wasserstein Manifold, by Hoang Phuc Hau Luu et al.
Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold
by Hoang Phuc Hau Luu, Hanlin Yu, Bernardo Williams, Marcelo Hartmann, Arto Klami
First submitted to arxiv on: 3 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Optimization in the Bures-Wasserstein space, connecting variational inference and Wasserstein gradient flows, has gained popularity. The Kullback-Leibler divergence can be written as the sum of negative entropy and potential energy, making forward-backward Euler a suitable method. Notably, the backward step has a closed-form solution, but the forward step is not exact due to intractable expectations. To address this, recent approaches have proposed using the Monte Carlo method for approximation, resulting in high variance and poor performance. We propose a novel variance-reduced estimator based on control variates, theoretically showing it has lower variance than Monte Carlo in scenarios of interest. Our results also demonstrate order-of-magnitude improvements over previous Bures-Wasserstein methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making computers learn better by using a new way to optimize functions. It connects two ideas: one from machine learning and another from math. The old way of doing this was not very good because it gave confusing results. We propose a new method that makes the computer learn faster and more accurately. This method uses a special trick called “control variates” that helps reduce errors. We show that our method works better than the old one in certain situations, which is important for making computers do useful tasks. |
Keywords
» Artificial intelligence » Inference » Machine learning » Optimization
