Summary of Exploring Token Pruning in Vision State Space Models, by Zheng Zhan et al.
Exploring Token Pruning in Vision State Space Models
by Zheng Zhan, Zhenglun Kong, Yifan Gong, Yushu Wu, Zichong Meng, Hangyu Zheng, Xuan Shen, Stratis Ioannidis, Wei Niu, Pu Zhao, Yanzhi Wang
First submitted to arxiv on: 27 Sep 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes an innovative approach to enhance the efficiency of State Space Models (SSMs) in vision tasks by introducing a novel token pruning method specifically designed for SSM-based vision models. By leveraging the unique computational characteristics of SSMs, the authors develop a pruning-aware hidden state alignment method and a token importance evaluation method adapted for SSM models. These techniques enable actual speedup with minimal impact on performance across different tasks. The authors demonstrate significant computation reduction with an 81.7% accuracy on ImageNet using PlainMamba-L3 model with a 41.6% reduction in FLOPs. This work provides valuable insights into understanding the behavior of SSM-based vision models for future research. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us understand how we can make computer programs better at looking at pictures. It’s about finding ways to make these programs use less energy and still get good results. The authors tried different methods, but they didn’t work well until they figured out a new way to do it. They came up with two special techniques that help the program decide which parts of the picture are most important. This allows the program to be faster and more efficient while still doing a good job. The authors tested their method on a big set of pictures and got great results, showing how much energy they can save. |
Keywords
» Artificial intelligence » Alignment » Pruning » Token
