Summary of Lpntk: Better Generalisation with Less Data Via Sample Interaction During Learning, by Shangmin Guo et al.
lpNTK: Better Generalisation with Less Data via Sample Interaction During Learning
by Shangmin Guo, Yi Ren, Stefano V.Albrecht, Kenny Smith
First submitted to arxiv on: 16 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research proposes a novel approach to improve the generalization of artificial neural networks (ANNs) by considering the impact of training data on learning. The study starts by analyzing the interaction between samples in supervised learning, finding that labels influence this interaction. This insight leads to the development of labelled pseudo Neural Tangent Kernel (lpNTK), which takes label information into account when measuring interactions between samples. The authors demonstrate that lpNTK converges to the empirical neural tangent kernel under certain assumptions and show how it can help understand learning phenomena, such as learning difficulty and forgetting events. Furthermore, they illustrate how lpNTK can be used to identify and remove poisoning training samples without harming generalization performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research helps us better understand how artificial neural networks learn from data. By looking at how different pieces of information interact with each other during training, the scientists developed a new way to measure this interaction called lpNTK. This tool can help explain why some things are easy or hard for the network to learn and even help remove bad training data that might hurt the network’s performance. |
Keywords
* Artificial intelligence * Generalization * Supervised
