Summary of Mitigating Quantization Errors Due to Activation Spikes in Glu-based Llms, by Jaewoo Yang et al.
Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs
by Jaewoo Yang, Hayun Kim, Younghoon Kim
First submitted to arxiv on: 23 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper investigates the challenges of activation quantization in GLU variants, widely used in feed-forward networks (FFNs) of modern large language models (LLMs), such as LLaMA family. The problem lies in severe local quantization errors caused by excessive magnitudes of activations in GLU variants, significantly degrading the performance of the quantized LLM. To address this issue, the authors propose two empirical methods: Quantization-free Module (QFeM) and Quantization-free Prefix (QFeP), to isolate activation spikes during quantization. The paper’s extensive experiments validate the effectiveness of these methods for activation quantization, especially with coarse-grained schemes, in latest LLMs with GLU variants. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper explores a problem in large language models that can reduce their performance. It finds that some parts of the model can get very big or small, which makes it hard to use them in lower-precision computers. The researchers propose two new ways to deal with this issue, which help improve the performance of these models when used in lower-precision computers. |
Keywords
» Artificial intelligence » Llama » Precision » Quantization
