Summary of Minimum Entropy Coupling with Bottleneck, by M.reza Ebrahimi et al.
Minimum Entropy Coupling with Bottleneck
by M.Reza Ebrahimi, Jun Chen, Ashish Khisti
First submitted to arxiv on: 29 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Information Theory (cs.IT)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed lossy compression framework operates under logarithmic loss, handling situations where reconstruction distribution diverges from source distribution. This is particularly relevant for joint compression and retrieval applications and scenarios involving distributional shifts due to processing. The framework extends the classical minimum entropy coupling by integrating a bottleneck, allowing controlled stochasticity in the coupling. The paper decomposes the Minimum Entropy Coupling with Bottleneck (MEC-B) into two optimization problems: Entropy-Bounded Information Maximization (EBIM) for the encoder and MEC for the decoder. A greedy algorithm is provided for EBIM with guaranteed performance, and the optimal solution near functional mappings is characterized, yielding significant theoretical insights. The paper also demonstrates practical application of MEC-B through Markov Coding Games (MCGs) under rate limits. Experiments showcase trade-offs between MDP rewards and receiver accuracy across various compression rates, highlighting the efficacy of the method compared to conventional compression baseline. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to compress information while still being able to retrieve it accurately. This is important for situations where the original information changes or moves away from its original form. The researchers develop this new framework by combining two existing methods: minimum entropy coupling and a bottleneck that adds some randomness to the process. They then break down their method into smaller parts and show how each part works, giving us a deeper understanding of what makes it tick. Finally, they test their approach using a game-like scenario where information is being transmitted and received in real-time. The results show that this new method can do better than existing methods in certain situations. |
Keywords
» Artificial intelligence » Decoder » Encoder » Optimization
