Summary of A Two-scale Complexity Measure For Deep Learning Models, by Massimiliano Datres et al.
A Two-Scale Complexity Measure for Deep Learning Models
by Massimiliano Datres, Gian Paolo Leonardi, Alessio Figalli, David Sutter
First submitted to arxiv on: 17 Jan 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel capacity measure called 2sED (statistical effective dimension) for statistical models, which is based on the effective dimension concept. This new quantity provides an upper bound for the generalization error under mild assumptions on the model. The authors demonstrate that 2sED correlates well with the training error through simulations on standard datasets and popular model architectures. Additionally, they develop a layerwise iterative approach to efficiently approximate 2sED from below, which enables them to tackle deep learning models with a large number of parameters. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The researchers developed a new way to measure how complex statistical models are. They called it 2sED (statistical effective dimension). This measurement helps us understand how well the model will work on new data that hasn’t been seen before. The authors tested their idea and found that it works well with different popular machine learning models and datasets. |
Keywords
* Artificial intelligence * Deep learning * Generalization * Machine learning
