Summary of Multi-epoch Learning with Data Augmentation For Deep Click-through Rate Prediction, by Zhongxiang Fan et al.
Multi-Epoch learning with Data Augmentation for Deep Click-Through Rate Prediction
by Zhongxiang Fan, Zhaocheng Liu, Jian Liang, Dongying Kong, Han Li, Peng Jiang, Shuang Li, Kun Gai
First submitted to arxiv on: 27 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Information Retrieval (cs.IR); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the one-epoch overfitting phenomenon in Click-Through Rate (CTR) models, where performance declines at the start of the second epoch. Despite extensive research, the efficacy of multi-epoch training remains unclear. The authors identify the overfitting of the embedding layer as the primary issue due to high-dimensional data sparsity. To address this, they introduce a novel Multi-Epoch learning with Data Augmentation (MEDA) framework suitable for both non-continual and continual learning scenarios. MEDA minimizes overfitting by reducing the dependency on subsequent training data or MLP layers, achieving data augmentation through varied embedding spaces. The authors conduct extensive experiments on public and business datasets, demonstrating the effectiveness of data augmentation and superiority over conventional single-epoch training. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at why some models stop working well when they’re updated with new information. It finds that this is often because the model becomes too good at remembering specific details instead of understanding the general patterns. To solve this problem, the authors created a new way to update models called MEDA (Multi-Epoch learning with Data Augmentation). This approach helps models learn from new information without forgetting what they already know. The authors tested MEDA on several different datasets and found that it works better than traditional methods. |
Keywords
* Artificial intelligence * Continual learning * Data augmentation * Embedding * Overfitting
