Summary of Model Compression and Efficient Inference For Large Language Models: a Survey, by Wenxiao Wang et al.
Model Compression and Efficient Inference for Large Language Models: A Survey
by Wenxiao Wang, Wei Chen, Yicong Luo, Yongliu Long, Zhengkai Lin, Liye Zhang, Binbin Lin, Deng Cai, Xiaofei He
First submitted to arxiv on: 15 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Performance (cs.PF)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the development of compression and efficient inference methods for large language models from an algorithmic perspective. The authors identify two key characteristics of these models: the high cost associated with fine-tuning or retraining after compression, and the emphasis on versatility and generalization over performance on a single task. To address these challenges, the paper proposes various algorithms, including quantization, pruning, distillation, and compact architecture design, as well as dynamic networks that prioritize tuning-free methods. The authors also provide an introduction to mature frameworks for efficient inference of large models, which can facilitate model deployment for users. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research aims to make large language models more usable on devices with limited resources. The problem is that these powerful models are too big and use too much memory and computing power. To fix this, scientists have developed ways to shrink these models or speed them up without losing their effectiveness. This paper looks at some of the best methods for doing this, including techniques that don’t require retraining the model after shrinking it. The goal is to create a system where users can easily deploy large language models on devices with limited resources. |
Keywords
* Artificial intelligence * Distillation * Fine tuning * Generalization * Inference * Pruning * Quantization
