Summary of A Rate-distortion View Of Uncertainty Quantification, by Ifigeneia Apostolopoulou et al.
A Rate-Distortion View of Uncertainty Quantification
by Ifigeneia Apostolopoulou, Benjamin Eysenbach, Frank Nielsen, Artur Dubrawski
First submitted to arxiv on: 16 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes Distance Aware Bottleneck (DAB), a novel approach to enriching deep neural networks with the ability to understand an input’s proximity to the training data. This property is essential for making reliable predictions in supervised learning. Unlike powerful probabilistic models like Gaussian Processes, deep neural networks typically lack this capability. The authors build upon prior information bottleneck methods and develop a codebook that stores compressed representations of all inputs seen during training. This allows the model to estimate uncertainty by calculating the distance between new examples and the codebook. The resulting model is easy to train and provides deterministic uncertainty estimates through a single forward pass. Notably, DAB achieves better out-of-distribution (OOD) detection and misclassification prediction compared to prior methods, including ensemble approaches, deep kernel Gaussian Processes, and standard information bottleneck techniques. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you’re trying to guess what kind of animal is in a photo. You’ve seen many animals before, but this one looks different. A “distance aware” model would help you figure out how similar or different this new animal is compared to the ones you’ve seen before. This paper introduces a new way to do just that using something called the Distance Aware Bottleneck (DAB) method. It’s like having a special dictionary that helps the model understand what makes things similar or different. This new approach can help the model make better guesses about things it hasn’t seen before, and it’s actually pretty easy to use. |
Keywords
» Artificial intelligence » Supervised
