Summary of Pruning One More Token Is Enough: Leveraging Latency-workload Non-linearities For Vision Transformers on the Edge, by Nick John Eliopoulos et al.
Pruning One More Token is Enough: Leveraging Latency-Workload Non-Linearities for Vision Transformers on the Edge
by Nick John Eliopoulos, Purvish Jajal, James C. Davis, Gaowen Liu, George K. Thiravathukal, Yung-Hsiang Lu
First submitted to arxiv on: 1 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores strategies for efficiently deploying vision transformers on edge devices for small workloads. Recent methods have reduced latency by removing or merging tokens with minimal accuracy loss. However, these approaches were not designed with edge device deployment in mind and do not leverage information about the latency-workload trends to improve efficiency. The authors address this shortcoming by identifying factors affecting ViT latency-workload relationships, determining a token pruning schedule based on non-linear latency-workload relationships, and proposing a training-free token pruning method utilizing this schedule. Experimental results demonstrate that their approach reduces latency by 9-26% compared to other methods, which can increase latency by 2-30%. The authors also show that their approach achieves ImageNet1K accuracy of 78.6%-84.5% while maintaining similar latency (within 5.2% or 7ms) across devices. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making computer vision models work better on smaller computers, like those used in smart cameras and self-driving cars. Right now, these models are too slow for small tasks, but new techniques can make them faster with only a tiny loss of accuracy. The authors found that the best way to do this is by using information about how fast the computer is and what it’s doing, rather than just removing or merging parts of the model. They tested their approach and showed that it works better than others in reducing latency while maintaining accuracy. |
Keywords
* Artificial intelligence * Pruning * Token * Vit
