Summary of One-step Early Stopping Strategy Using Neural Tangent Kernel Theory and Rademacher Complexity, by Daniel Martin Xavier and Ludovic Chamoin and Jawher Jerray and Laurent Fribourg
One-Step Early Stopping Strategy using Neural Tangent Kernel Theory and Rademacher Complexity
by Daniel Martin Xavier, Ludovic Chamoin, Jawher Jerray, Laurent Fribourg
First submitted to arxiv on: 27 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Systems and Control (eess.SY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an analytical estimation of the optimal early stopping time for neural networks, which involves the initial training error vector and the eigenvalues of the neural tangent kernel. This approach yields an upper bound on the population loss, suitable for underparameterized scenarios where the number of parameters is moderate compared to the number of data. The method is demonstrated using an example of a neural network simulating MPC control of a Van der Pol oscillator. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how to stop training a neural network early without overfitting. They develop a way to estimate when it’s best to stop training, based on some initial error and the “neural tangent kernel”. This helps with underparameterized models where there aren’t enough parameters to fit all the data. The method is shown to work well in controlling an oscillator. |
Keywords
» Artificial intelligence » Early stopping » Neural network » Overfitting
