Summary of Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples For Deep Learning Models, by Anshuman Chhabra et al.
Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models
by Anshuman Chhabra, Bo Li, Jian Chen, Prasant Mohapatra, Hongfu Liu
First submitted to arxiv on: 6 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers aim to develop a more efficient method for identifying training samples that negatively impact model performance. They build on influence functions, a widely used tool for assessing the effect of individual training data points on model predictions. However, calculating influence functions can be computationally expensive, especially when working with large deep models. The authors propose an alternative approach that combines outlier gradient detection and influence functions to identify detrimental samples without requiring Hessian matrix calculations. They validate their method using synthetic datasets and demonstrate its effectiveness in detecting mislabeled samples in vision models and selecting data points for improving performance of natural language processing transformer models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about finding the bad training data that makes AI models do worse. Researchers are trying to make it easier to find these problems without needing powerful computers. They’re building on a method called influence functions, which helps figure out how individual pieces of data affect the model’s predictions. The new approach combines two ideas together and works faster than the original method. It was tested with fake data and worked well in finding bad data for AI models that look at pictures or understand language. |
Keywords
» Artificial intelligence » Natural language processing » Transformer
