Summary of Kernelwarehouse: Rethinking the Design Of Dynamic Convolution, by Chao Li et al.
KernelWarehouse: Rethinking the Design of Dynamic Convolution
by Chao Li, Anbang Yao
First submitted to arxiv on: 12 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A new approach to dynamic convolution is proposed, which leverages parameter dependencies within and across layers of a ConvNet. This method, called KernelWarehouse, redefines the concepts of kernels, assembling kernels, and attention functions to achieve superior performance while being more parameter-efficient than traditional dynamic convolution. The authors demonstrate the effectiveness of KernelWarehouse on various ConvNet architectures and datasets, including ImageNet and MS-COCO, as well as its applicability to Vision Transformers, which can even reduce model size while improving accuracy. For example, KernelWarehouse with n=4 achieves a 5.61% absolute top-1 accuracy gain on the ResNet18 backbone. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Dynamic convolution is a new way of doing things in machine learning. It’s like having many special filters that help your computer see and recognize things. The usual way of doing this makes it hard to use lots of these filters, but now there’s a better way called KernelWarehouse. This helps computers understand pictures and videos even better, and can even make the computers themselves smaller while still being good at recognizing things. |
Keywords
» Artificial intelligence » Attention » Machine learning » Parameter efficient
