Summary of (implicit) Ensembles Of Ensembles: Epistemic Uncertainty Collapse in Large Models, by Andreas Kirsch
(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models
by Andreas Kirsch
First submitted to arxiv on: 4 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: This paper investigates the phenomenon of epistemic uncertainty collapse in deep learning models as model complexity increases. Contrary to assumptions, larger models do not always offer better uncertainty quantification. The authors propose that this stems from implicit ensembling within large models and demonstrate empirically across various architectures, including ResNets and Vision Transformers. They also provide theoretical justification for these phenomena and explore their implications for uncertainty estimation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: This paper looks at how deep learning models handle unknown or unexpected situations. It finds that bigger models don’t always do a better job of figuring out what’s going on when things get weird. The authors think this is because big models are actually doing a lot of internal “testing” to try to make sense of things, and they show some interesting examples to support their idea. They also explain why this matters for important applications like self-driving cars. |
Keywords
* Artificial intelligence * Deep learning
