Summary of Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum, by Tin Sum Cheng and Aurelien Lucchi and Anastasis Kratsios and David Belius
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
by Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, David Belius
First submitted to arxiv on: 2 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers develop new bounds for kernel matrices, which they use to improve existing test error bounds for kernel ridgeless regression (KRR) in the over-parameterized regime. The new bounds are non-asymptotic and applicable to fixed input dimensions. For kernels with polynomial spectral decay, the results match previous work; for exponential decay, the bounds are novel and non-trivial. The study also explores phenomena such as tempered overfitting and catastrophic overfitting under sub-Gaussian design assumptions, filling a gap in the literature. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper studies how to make predictions more accurate when using kernel ridgeless regression (KRR) with lots of features. It finds that some types of kernel matrices can cause problems if there are too many features. The researchers come up with new rules to help predict how well KRR will work, which can be useful for making machines learn from data. |
Keywords
* Artificial intelligence * Overfitting * Regression
