Summary of Ostr-darts: Differentiable Neural Architecture Search Based on Operation Strength, by Le Yang et al.
OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength
by Le Yang, Ziwei Zheng, Yizeng Han, Shiji Song, Gao Huang, Fan Li
First submitted to arxiv on: 22 Sep 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an alternative approach to differentiable architecture search (DARTS), a technique used to find high-performance neural architectures. The existing DARTS method consists of two steps: optimizing a supernet that contains mixed operations via gradient descent, and selecting the most contributing operations to build the final architecture. While DARTS improves efficiency, it suffers from the degeneration issue, where architectures deteriorate over time. The authors attribute this issue to the failure of supernet optimization, but propose a novel criterion based on operation strength to estimate importance. They demonstrate that using this criterion without modifying supernet optimization can effectively address degeneration, and conduct experiments on NAS-Bench-201 and DARTS search spaces. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about finding the best way to design neural networks. The current method, called DARTS, has a problem where it makes the designs worse over time. The researchers looked at why this happens and found that it’s because of how they choose which operations to use in the final design. They came up with a new way to do this, using something called “operation strength”, which helps them find better designs without making them worse. |
Keywords
» Artificial intelligence » Gradient descent » Optimization
