Summary of Aux-nas: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost, by Yuan Gao et al.
Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost
by Yuan Gao, Weizhong Zhang, Wenhan Luo, Lin Ma, Jin-Gang Yu, Gui-Song Xia, Jiayi Ma
First submitted to arxiv on: 9 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel architecture-based approach to auxiliary learning, which aims to boost the performance of a primary task while maintaining a single-task inference cost. The method, called Aux-NAS, is a Neural Architecture Search (NAS) problem that initializes bi-directional connections between the primary and auxiliary tasks. The NAS optimization converges to an architecture with only single-side primary-to-auxiliary connections, which can be removed during primary task inference. The approach can be combined with optimization-based auxiliary learning methods. Experimental results on six tasks using VGG, ResNet, and ViT backbones demonstrate promising performance improvements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper finds a new way to use extra information from another task to make the main task better. They create a special network structure that connects the two tasks together during training, but not when it’s time to make predictions on just one of them. This helps the main task get better without making it take longer or using more computer power. The results show that this works well for six different tasks. |
Keywords
» Artificial intelligence » Inference » Optimization » Resnet » Vit
