Summary of Rilq: Rank-insensitive Lora-based Quantization Error Compensation For Boosting 2-bit Large Language Model Accuracy, by Geonho Lee et al.
RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracy
by Geonho Lee, Janghwan Lee, Sukjin Hong, Minsoo Kim, Euijai Ahn, Du-Seong Chang, Jungwook Choi
First submitted to arxiv on: 2 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach called RILQ (Rank-Insensitive LoRA-based Quantization Error Compensation) is proposed to address the limitation of LoRA-based quantization error compensation (LQEC) in sub-4-bit scenarios. By understanding the fundamental limitations and rank-insensitive nature of model-wise activation discrepancy loss, RILQ adjusts adapters cooperatively across layers, enabling robust error compensation with low-rank adapters. This approach achieves consistent improvements in 2-bit quantized inference on LLaMA-2 and LLaMA-3, as well as enhanced accuracy in task-specific fine-tuning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary RILQ is a new way to make language models work better when they’re being used with less memory. The problem is that when we make the model smaller, it doesn’t work as well anymore. RILQ helps solve this problem by making small changes to how the model works. It does this by looking at how different parts of the model are working and adjusting them to help each other. This makes the model work better even when it’s being used with very little memory. |
Keywords
» Artificial intelligence » Fine tuning » Inference » Llama » Lora » Quantization
