Summary of Compression For Better: a General and Stable Lossless Compression Framework, by Boyang Zhang et al.
Compression for Better: A General and Stable Lossless Compression Framework
by Boyang Zhang, Daning Cheng, Yunquan Zhang, Fangmin Liu, Wenguang Chen
First submitted to arxiv on: 9 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed LossLess Compression (LLC) framework aims to stabilize and losslessly compress neural networks, reducing complexity and enhancing efficiency without sacrificing performance due to compression errors. The key challenge is defining the boundaries for lossless compression to minimize model loss. Currently, there is no systematic approach to determining this error boundary or understanding its impact on model performance. LLC delineates the compression neighborhood and higher-order analysis boundaries through total differential, specifying the error range within which a model can be compressed without loss. To verify LLC’s effectiveness, various compression techniques are applied, including quantization and decomposition. Quantization is reformulated as a grouped knapsack problem within the lossless neighborhood, achieving lossless quantization while improving computational efficiency. Decomposition addresses the approximation problem under low-rank constraints, automatically determining the rank for each layer and producing lossless low-rank models. Extensive experiments on multiple neural network architectures on different datasets show that LLC can effectively achieve lossless model compression without fancy tricks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research tries to make it possible to shrink big neural networks without making them worse or losing their important abilities. The problem is figuring out how much a network can be shrunk before it starts to get worse. Right now, there’s no good way to do this. So, the team created a new framework called LLC that helps figure out where the boundaries are for shrinking a network without losing its power. They tested this on many different types of networks and datasets and found that it works really well. |
Keywords
» Artificial intelligence » Model compression » Neural network » Quantization
