Summary of Salnas: Efficient Saliency-prediction Neural Architecture Search with Self-knowledge Distillation, by Chakkrit Termritthikun et al.
SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation
by Chakkrit Termritthikun, Ayaz Umer, Suwichaya Suwanwimolkul, Feng Xia, Ivan Lee
First submitted to arxiv on: 29 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Neural Architecture Search (NAS) framework for saliency prediction utilizes a supernet, SalNAS, which integrates dynamic convolution into an encoder-decoder architecture. To address the issue of generalization in SalNAS, a self-knowledge distillation approach, Self-KD, is introduced. This approach trains the student model with weighted average information from the teacher model, allowing for efficient training without computing gradients. The resulting SalNAS outperforms state-of-the-art models on seven benchmark datasets while being lightweight. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Saliency prediction has improved thanks to deep convolutional neural networks. However, creating these networks can be time-consuming and require expertise. A new Neural Architecture Search (NAS) framework is proposed to solve this issue. The framework uses a supernet with dynamic convolution and a self-knowledge distillation approach to train the model. This allows for efficient training without needing complex computations. The resulting model performs well on multiple datasets. |
Keywords
» Artificial intelligence » Encoder decoder » Generalization » Knowledge distillation » Student model » Teacher model
