Summary of Unveiling Multiple Descents in Unsupervised Autoencoders, by Kobi Rahimi et al.
Unveiling Multiple Descents in Unsupervised Autoencoders
by Kobi Rahimi, Yehonathan Refael, Tom Tirer, Ofir Lindenbaum
First submitted to arxiv on: 17 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The study investigates the phenomenon of double descent in unsupervised learning, particularly in linear and nonlinear autoencoders (AEs). It is found that while double descent does not occur in linear AEs, both double and triple descent can be observed with nonlinear AEs across various data models and architectural designs. The effects of partial sample and feature noise are examined, highlighting the importance of bottleneck size in influencing the double descent curve. Through extensive experiments on synthetic and real datasets, model-wise, epoch-wise, and sample-wise double descent is uncovered, indicating that over-parameterized models improve reconstruction and enhance performance in downstream tasks such as anomaly detection and domain adaptation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how well autoencoders (AI models) work when they’re not told what the answers are. Normally, these models get better as they learn, but sometimes they can actually get worse before getting better again. This is called “double descent”. The researchers found that this happens with some types of AI models, but not others. They also found that making small mistakes in the data can make a big difference. By testing different kinds of data and AI models, they discovered that some models do better than others when it comes to finding patterns in noisy or changing data. |
Keywords
* Artificial intelligence * Anomaly detection * Domain adaptation * Unsupervised
