Summary of Oac: Output-adaptive Calibration For Accurate Post-training Quantization, by Ali Edalati (1) et al.
OAC: Output-adaptive Calibration for Accurate Post-training Quantization
by Ali Edalati, Alireza Ghaffari, Masoud Asgharian, Lu Hou, Boxing Chen, Vahid Partovi Nia
First submitted to arxiv on: 23 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to compressing Large Language Models (LLMs) is proposed, addressing the issue of rapidly expanding model size. By incorporating the model output in the calibration process, Output-adaptive Calibration (OAC) aims to reduce the accuracy drop often seen in low-precision quantization. This method builds upon Post-training Quantization (PTQ) techniques, which have been shown to be effective in compressing LLMs while avoiding expensive re-training. OAC uses output-adaptive Hessians to update weight matrices and detect salient weights, achieving state-of-the-art performance even at extreme low-precision quantization levels. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large Language Models are getting bigger and bigger, which is a problem because it takes up too much computer power. Scientists have been trying to find ways to make them smaller without losing their abilities. One way they do this is by “quantizing” the model, or changing how the computer stores the information. They use something called Post-training Quantization (PTQ) and it works pretty well. But sometimes when they do this, the model gets a little worse at doing its job. The new method, Output-adaptive Calibration (OAC), tries to fix this by making sure the model is still good at what it does even when it’s being used in a low-precision way. |
Keywords
» Artificial intelligence » Precision » Quantization
