Summary of Partially Frozen Random Networks Contain Compact Strong Lottery Tickets, by Hikari Otsuka et al.
Partially Frozen Random Networks Contain Compact Strong Lottery Tickets
by Hikari Otsuka, Daiki Chijiwa, Ángel López García-Arias, Yasuyuki Okoshi, Kazushi Kawamura, Thiem Van Chu, Daichi Fujiki, Susumu Takeuchi, Masato Motomura
First submitted to arxiv on: 20 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract proposes a method to reduce the memory size of “strong lottery tickets” (SLTs) in machine learning models without sacrificing their accuracy. The authors demonstrate that by freezing a subset of initial weights, they can achieve better accuracy-to-model size trade-offs compared to traditional methods. Specifically, they show that freezing 70% of a ResNet on ImageNet provides 3.3 times compression and raises accuracy by up to 14.12 points. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This method is useful for reducing the memory footprint of machine learning models without sacrificing their performance. By freezing some weights and pruning others, the authors achieve better results than traditional methods that only prune or randomize weights. This technique could be applied to a wide range of models and applications where memory is limited. |
Keywords
* Artificial intelligence * Machine learning * Pruning * Resnet
