Summary of Updp: a Unified Progressive Depth Pruner For Cnn and Vision Transformer, by Ji Liu et al.
UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer
by Ji Liu, Dehua Tang, Yuanxian Huang, Li Zhang, Xiaocheng Zeng, Dong Li, Mingjie Lu, Jinzhang Peng, Yu Wang, Fan Jiang, Lu Tian, Ashish Sirasao
First submitted to arxiv on: 12 Jan 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the issue of traditional channel-wise pruning methods struggling to effectively prune efficient CNN models with depth-wise convolutional layers and certain efficient modules, such as popular inverted residual blocks. The authors propose a novel depth pruning method for efficient models that incorporates a block pruning strategy and progressive training method. Additionally, they extend their pruning method to vision transformer models. Experimental results demonstrate that the proposed method consistently outperforms existing depth pruning methods across various pruning configurations, achieving state-of-the-art pruning performance on both CNN and vision transformer models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps make computer vision models more efficient by finding ways to remove parts of them without losing their ability to perform well. The authors try to solve the problem that usual methods for removing channels from a model don’t work well with certain types of layers, like those used in popular “efficient” models. They come up with a new way to remove these layers and train the remaining parts to still work well. This helps both traditional computer vision models and newer transformer-based models. |
Keywords
» Artificial intelligence » Cnn » Pruning » Transformer » Vision transformer
