Summary of Outlier-aware Training For Low-bit Quantization Of Structural Re-parameterized Networks, by Muqun Niu et al.
Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks
by Muqun Niu, Yuan Ren, Boyu Li, Chenchen Ding
First submitted to arxiv on: 11 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed structural re-parameterized network, RepVGG, combines simplicity and high accuracy by revitalizing VGG-like networks. A novel training method, Outlier Aware Batch Normalization (OABN), is introduced to address the challenges posed by outliers in SR networks’ weights. Additionally, a clustering-based non-uniform quantization framework for Quantization-Aware Training (QAT) named ClusterQAT is developed to enhance inference accuracy even when bitwidths fall below 8. By integrating OABN with ClusterQAT, the performance of RepVGG is significantly improved. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper proposes a new way to design and train Convolutional Neural Networks (CNNs). The idea is to make simple networks work just as well as more complicated ones. To do this, they create a special type of network called RepVGG that can be trained in one way but works differently during testing. This creates some problems, like uneven weights, which they try to fix with a new training method called OABN. They also develop a way to make the network work well even when it has limited computing power (called quantization). By combining these two ideas, they can make their simple network work just as well as more complicated ones. |
Keywords
* Artificial intelligence * Batch normalization * Clustering * Inference * Quantization
