Summary of Aggressive Post-training Compression on Extremely Large Language Models, by Zining Zhang et al.
Aggressive Post-Training Compression on Extremely Large Language Models
by Zining Zhang, Yao Chen, Bingsheng He, Zhenjie Zhang
First submitted to arxiv on: 30 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed novel network pruning technology enables the compression of prevailing Large Language Models (LLMs) within a couple of hours while maintaining a relatively small accuracy loss. The approach utilizes over 0.7 sparsity and less than 8 bits of quantization, making it possible to deploy LLMs on personal computers and mobile devices. Experimental evaluations demonstrate the effectiveness and potential for practical deployment of this technology. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to make Large Language Models smaller so they can be used on personal devices like phones and computers has been developed. These models are getting bigger and harder to use because they take up too much space and power. The solution is a special kind of pruning that gets rid of some parts of the model without making it worse. This makes it possible to use these language models anywhere, which could have big impacts on how we communicate. |
Keywords
» Artificial intelligence » Pruning » Quantization
