Summary of Quantization Of Large Language Models with An Overdetermined Basis, by Daniil Merkulov et al.
Quantization of Large Language Models with an Overdetermined Basis
by Daniil Merkulov, Daria Cherniuk, Alexander Rudikov, Ivan Oseledets, Ekaterina Muravleva, Aleksandr Mikhalev, Boris Kashin
First submitted to arxiv on: 15 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Kashin Quantization algorithm decomposes vectors, matrices, or tensors into two factors, allowing for efficient data compression. By leveraging the principles of Kashin representation, the algorithm maintains a small infinity norm and constrained norms when multiplied by an orthogonal matrix. The decomposed entries are well-concentrated around peaks, enabling replacement with centroids for quantization purposes. Theoretical properties and rigorous evaluations on next-word prediction tasks and text classification show that Kashin Quantization achieves competitive or superior model performance while ensuring data compression. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Kashin Quantization is a new way to compress big data files. It works by breaking down the data into two parts, like separating light from darkness. The first part stays small, and the second part gets smaller when mixed with special numbers called orthogonal matrices. This makes it easy to replace many of the original values with simpler ones called centroids. Scientists tested this method on tasks like predicting what comes next in a sentence or classifying text, and found that it works well and saves space. |
Keywords
» Artificial intelligence » Quantization » Text classification
