Summary of Entprop: High Entropy Propagation For Improving Accuracy and Robustness, by Shohei Enomoto
EntProp: High Entropy Propagation for Improving Accuracy and Robustness
by Shohei Enomoto
First submitted to arxiv on: 29 May 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers explore ways to improve the performance of deep neural networks (DNNs) by increasing their ability to generalize to out-of-distribution domains. The authors propose a technique called disentangled learning with mixture distribution via auxiliary batch normalization layers (ABNs), which treats clean and transformed samples as different domains, allowing DNNs to learn better features from mixed domains. They also introduce two techniques – data augmentation and free adversarial training – that increase entropy and bring the sample further away from the in-distribution domain, without requiring additional training costs. The proposed method, high entropy propagation (EntProp), achieves higher standard accuracy and robustness with a lower training cost than baseline methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making computers learn better. Right now, they’re really good at recognizing things when they look normal, but they struggle when things are different or unusual. The researchers want to improve this by teaching the computer to recognize patterns in all kinds of situations, not just normal ones. They came up with a way to do this using something called disentangled learning and special layers called ABNs. They also found two other ways to make the computer learn even better without needing more training – data augmentation and free adversarial training. This new method is called EntProp and it does really well in tests, especially when working with small datasets. |
Keywords
» Artificial intelligence » Batch normalization » Data augmentation
