Summary of Extreme Compression Of Large Language Models Via Additive Quantization, by Vage Egiazarian et al.
Extreme Compression of Large Language Models via Additive Quantization
by Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, Dan Alistarh
First submitted to arxiv on: 11 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents an innovative approach to compressing large language models (LLMs) using Multi-Codebook Quantization (MCQ). The goal is to develop a scheme that can efficiently execute LLMs on end-user devices. The authors propose AQLM, a method that builds upon the classic Additive Quantization (AQ) technique and introduces two key innovations: learned additive quantization of weight matrices and joint optimization of codebook parameters across transformer blocks. The resulting algorithm achieves state-of-the-art performance in compressing LLMs to extremely low bit counts (2-3 bits per parameter), while also being practical and efficient. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is all about making big language models work on small devices, like your phone or computer. It’s a tricky problem because these models are super powerful but take up a lot of memory and processing power. The authors came up with a new way to shrink these models down without losing their ability to understand and generate human-like text. Their method is called AQLM and it works by cleverly adjusting the model’s internal workings to use fewer bits of information. This means the model can be stored in a much smaller space, making it possible to run on devices with limited resources. |
Keywords
* Artificial intelligence * Optimization * Quantization * Transformer
