Summary of Towards Meta-pruning Via Optimal Transport, by Alexander Theus et al.
Towards Meta-Pruning via Optimal Transport
by Alexander Theus, Olin Geimer, Friedrich Wicke, Thomas Hofmann, Sotiris Anagnostidis, Sidak Pal Singh
First submitted to arxiv on: 12 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents Intra-Fusion, a novel approach that redefines structural pruning of neural networks. Unlike conventional methods that focus on identifying important neurons, Intra-Fusion uses model fusion and Optimal Transport to arrive at a more effective sparse model representation. This approach achieves significant accuracy recovery without the need for fine-tuning, making it an efficient tool for neural network compression. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Intra-Fusion is a new way to make neural networks smaller while keeping them just as good. Normally, when we shrink a neural network, some parts become less important and we get rid of those. But this method doesn’t work well because we have to fix the network again afterwards. Intra-Fusion changes how we shrink the network by using something called model fusion and Optimal Transport. This makes it better at keeping the accuracy we need without needing to make any more changes. |
Keywords
* Artificial intelligence * Fine tuning * Neural network * Pruning
