Summary of Flatnas: Optimizing Flatness in Neural Architecture Search For Out-of-distribution Robustness, by Matteo Gambella et al.
FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness
by Matteo Gambella, Fabrizio Pittorino, Manuel Roveri
First submitted to arxiv on: 29 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel Neural Architecture Search (NAS) solution called FlatNAS combines a figure of merit based on robustness to weight perturbations with single NN optimization using Sharpness-Aware Minimization (SAM). FlatNAS is the first NAS procedure to explore flat regions in the loss landscape of NNs, optimizing performance on in-distribution data, out-of-distribution (OOD) robustness, and constraining architecture parameters. Unlike OOD-focused studies, FlatNAS evaluates the impact of architectures on OOD generalization, crucial for real-world applications. By using only in-distribution data, FlatNAS achieves a good trade-off between performance, OOD robustness, and parameter count. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary FlatNAS is a new way to design neural networks that work well even when they’re given messed-up input data. It’s like finding the right recipe for baking a cake – you need to find the perfect combination of ingredients and cooking time. FlatNAS does this by searching through lots of different network designs, looking for the ones that do well on normal data and also do well on weird data. The goal is to make networks that are robust, or able to withstand mistakes in their input. This is important because real-world applications often involve messy or corrupted data. |
Keywords
* Artificial intelligence * Generalization * Optimization * Sam
