Summary of Adversarial Training Of Two-layer Polynomial and Relu Activation Networks Via Convex Optimization, by Daniel Kuelbs et al.
Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimization
by Daniel Kuelbs, Sanjay Lall, Mert Pilanci
First submitted to arxiv on: 22 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the critical issue of training neural networks that are resilient to adversarial attacks. Building upon recent work reformulating two-layer ReLU and polynomial activation network training as convex programs, the authors develop a novel convex semidefinite program (SDP) for adversarially training two-layer polynomial activation networks. The SDP is shown to achieve the same globally optimal solution as its non-convex counterpart. Experimental results demonstrate that the convex SDP improves robust test accuracy against _attacks on multiple datasets compared to the original convex training formulation. Furthermore, the paper presents scalable implementations of adversarial training for two-layer polynomial and ReLU networks, compatible with popular machine learning libraries and GPU acceleration. By retraining the final layers of a Pre-Activation ResNet-18 model using both polynomial and ReLU activations, the authors demonstrate the practical utility of convex adversarial training on large-scale problems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making sure artificial intelligence systems are safe from being tricked by bad data. The researchers take inspiration from recent work that makes complicated math problems easier to solve. They create a new way to train neural networks so they can withstand attacks that try to make them do the wrong thing. This new method does better than previous methods on several tests and is fast enough to be used with big datasets. The authors also show that this approach works well when applied to a real-world problem, making it a useful tool for building safe AI systems. |
Keywords
» Artificial intelligence » Machine learning » Relu » Resnet
