Summary of Towards Accurate Post-training Quantization For Reparameterized Models, by Luoming Zhang et al.
Towards Accurate Post-training Quantization for Reparameterized Models
by Luoming Zhang, Yefei He, Wen Fei, Zhenyu Lou, Weijia Wu, YangWei Ying, Hong Zhou
First submitted to arxiv on: 25 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Model reparameterization is a widely accepted technique for improving inference speed without compromising performance. However, current Post-training Quantization (PTQ) methods often lead to significant accuracy degradation when applied to reparameterized models. To address this issue, we propose RepAPQ, a novel framework that preserves the accuracy of quantized reparameterization models. The framework utilizes Mean Absolute Error (MAE) instead of Mean Squared Error (MSE), which mitigates the influence of outliers on quantization parameters. RepAPQ comprises two main components: Quantization Protecting Reparameterization and Across-block Calibration. The code is available at this URL. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Reparameterized models are designed to speed up inference without sacrificing accuracy. But, current Post-training Quantization (PTQ) methods can actually make things worse! To fix this, we created a new framework called RepAPQ. It works by using Mean Absolute Error (MAE) instead of the usual Mean Squared Error (MSE). This helps get rid of outliers that mess with quantization parameters. RepAPQ has two parts: one that protects reparameterization and another that calibrates across blocks. We tested it on different models and tasks, and it worked really well! It’s even better than previous methods by about 1% for 8-bit PTQ and 2% for 6-bit PTQ. |
Keywords
» Artificial intelligence » Inference » Mae » Mse » Quantization
