Summary of Deep Generative Models Through the Lens Of the Manifold Hypothesis: a Survey and New Connections, by Gabriel Loaiza-ganem et al.
Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections
by Gabriel Loaiza-Ganem, Brendan Leigh Ross, Rasa Hosseinzadeh, Anthony L. Caterini, Jesse C. Cresswell
First submitted to arxiv on: 3 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores the relationship between deep generative models (DGMs) and the manifold hypothesis, investigating why some models succeed or fail when learning distributions on unknown low-dimensional manifolds. The study focuses on understanding the reasons behind the empirical superiority of certain DGMs, such as diffusion models and generative adversarial networks, over others like variational autoencoders and normalizing flows. To achieve this, the authors conduct a comprehensive survey of DGMs from the manifold lens perspective, making two novel contributions. First, they formally prove that numerical instability is inevitable when modeling data with low intrinsic dimension in high ambient dimensions. Second, they show that DGMs on learned representations of autoencoders can be interpreted as approximately minimizing Wasserstein distance, which helps justify their outstanding empirical results. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research looks at how deep learning models work and why some are better than others at creating new data that looks like real data. The study finds that some models do a great job of generating new data because they’re designed to learn from low-dimensional patterns, even if we don’t know what those patterns are. By understanding these patterns, the authors can develop better models that create more realistic data. |
Keywords
* Artificial intelligence * Deep learning * Diffusion
