Summary of Lnpt: Label-free Network Pruning and Training, by Jinying Xiao et al.
LNPT: Label-free Network Pruning and Training
by Jinying Xiao, Ping Li, Zhe Tang, Jie Nie
First submitted to arxiv on: 19 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores methods for deploying neural networks on resource-constrained smart devices while maintaining generalization performance. By retaining weights conducive to generalization, pruned networks can be accommodated on these devices. However, existing metrics for determining the pruned structures in advance are inconsistent with generalization during training processes. The authors introduce the concept of the learning gap, which accurately correlates with generalization, and propose a novel learning framework called LNPT that enables online guidance for network pruning and learning on smart devices using unlabeled data. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about how to make neural networks work well on small devices like smartphones. It’s important because we want to use these networks to do cool things on our phones, but they need to be able to learn from the data on our devices without needing a lot of computer power. The problem is that we don’t know ahead of time which parts of the network are most important for learning and can be safely removed (or “pruned”) to make it work better on small devices. This paper figures out a new way to measure how well the network will learn by looking at the patterns in its early layers, and shows that this approach works better than other methods. |
Keywords
* Artificial intelligence * Generalization * Pruning
