Summary of Towards Adversarially Robust Dataset Distillation by Curvature Regularization, By Eric Xue et al.
Towards Adversarially Robust Dataset Distillation by Curvature Regularization
by Eric Xue, Yijiang Li, Haoyang Liu, Peiran Wang, Yifan Shen, Haohan Wang
First submitted to arxiv on: 15 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: Dataset distillation (DD) allows datasets to be condensed into smaller sizes while preserving the rich information, enabling models trained on these distilled datasets to achieve comparable accuracy while reducing computational loads. Recent research has focused on improving model accuracy on distilled datasets. This paper explores a new perspective of DD by studying how to embed adversarial robustness in distilled datasets. The authors propose a method that incorporates curvature regularization into the distillation process, achieving better robustness with less computation overhead than standard adversarial training. Experimental results show that this approach outperforms standard adversarial training on accuracy and robustness while reducing computational costs. Moreover, the proposed method can generate robust distilled datasets that withstand various adversarial attacks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: Dataset distillation is a way to make large datasets smaller without losing important information. This helps models learn faster and use less computer power. Researchers have been trying to improve how well models perform on these smaller datasets. In this paper, scientists explore a new idea: making sure the distilled dataset is also robust against fake data that tries to trick the model. They developed a method that works with curvature regularization and proved it’s better than other methods at both accuracy and robustness while using less computer power. The results show that their approach can create datasets that are resistant to different types of attacks. |
Keywords
* Artificial intelligence * Distillation * Regularization
