Summary of Low-bit Quantization Favors Undertrained Llms: Scaling Laws For Quantized Llms with 100t Training Tokens, by Xu Ouyang et al.
Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, Dong Yu
First submitted to arxiv on: 26 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel discovery is made regarding the effects of low-bit quantization on large language models. It’s found that larger models or those with fewer training tokens are less affected by quantization-induced degradation, whereas smaller models with extensive training suffer more. To better understand this trend, researchers studied over 1,500 checkpoints of varying model sizes and training levels, developing scaling laws to explain the relationship between quantization and factors such as training tokens, model size, and bit width. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large language models can become smaller and faster by using fewer bits for calculations. However, this can also make them less accurate. Researchers found that bigger models or those that didn’t train as much are more resistant to this problem. On the other hand, smaller models that trained a lot are more affected. To figure out why this is happening, scientists looked at over 1,500 versions of these models with different sizes and training levels. |
Keywords
» Artificial intelligence » Quantization » Scaling laws
