Summary of Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxations, by Stefan Balauca et al.
Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxations
by Stefan Balauca, Mark Niklas Müller, Yuhao Mao, Maximilian Baader, Marc Fischer, Martin Vechev
First submitted to arxiv on: 11 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Training neural networks to achieve high certified accuracy against adversarial examples remains an open challenge in machine learning. Certification methods can leverage tight convex relaxations to compute bounds, but surprisingly, these methods can perform worse than looser relaxations during training. Prior work hypothesized that this phenomenon is caused by the discontinuity and perturbation sensitivity of the loss surface induced by tighter relaxations. In this paper, we theoretically show that Gaussian Loss Smoothing (GLS) can alleviate these issues. We confirm this empirically by instantiating GLS with two variants: a zeroth-order optimization algorithm called PGPE, which allows training with non-differentiable relaxations, and a first-order optimization algorithm called RGS, which requires gradients of the relaxation but is much more efficient than PGPE. Our results show that combining these methods with tight relaxations surpasses state-of-the-art methods when training on the same network architecture for many settings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to train a computer program (like a neural network) to be super good at recognizing things, like pictures or words. But there’s a problem – someone could easily trick it by adding fake information, making it think something is real when it’s not! To solve this issue, scientists have been working on creating methods that can “certify” whether the program is correct or not. In this paper, they show that using a technique called Gaussian Loss Smoothing (GLS) can help with this problem. They tested two different versions of GLS and found that it works really well when combined with tight relaxations, even beating other top methods! |
Keywords
* Artificial intelligence * Machine learning * Neural network * Optimization
