Summary of Shedding More Light on Robust Classifiers Under the Lens Of Energy-based Models, by Mujtaba Hussain Mirza et al.
Shedding More Light on Robust Classifiers under the lens of Energy-based Models
by Mujtaba Hussain Mirza, Maria Rosaria Briglia, Senad Beadini, Iacopo Masi
First submitted to arxiv on: 8 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a novel approach to understanding adversarial training (AT) by reinterpreting it as Energy-based Model (EBM). The authors analyze the energy landscape during AT and find that untargeted attacks generate images with lower energies than original data, while targeted attacks have higher energies. This insight leads to new theoretical and practical results, including the discovery of three phases in AT dynamics, overfitting in the third phase, and the ability of TRADES to alleviate overfitting by aligning natural and adversarial energies. The authors also propose a novel sample weighting scheme, Weighted Energy Adversarial Training (WEAT), which achieves state-of-the-art robust accuracy on multiple benchmarks. Furthermore, they show that robust classifiers have varying generative capabilities, which can be improved with a simple method. The code to reproduce the results is available online. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper helps us understand how adversarial training works by looking at it in a new way. It shows that when we try to make models more robust to attacks, the model’s “energy” changes. This energy is lower when we’re trying to trick the model and higher when we’re targeting specific parts of an image. The paper also finds that some ways of training models can actually help them avoid getting too good at recognizing certain patterns, which is useful for making sure they don’t get overfit. Additionally, it suggests a new way of weighting data points during training to make models even more robust. This could lead to better performance on tasks like image recognition. |
Keywords
* Artificial intelligence * Energy based model * Overfitting
