Summary of Entropic Distribution Matching in Supervised Fine-tuning Of Llms: Less Overfitting and Better Diversity, by Ziniu Li et al.
Entropic Distribution Matching in Supervised Fine-tuning of LLMs: Less Overfitting and Better Diversity
by Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Ruoyu Sun, Zhi-Quan Luo
First submitted to arxiv on: 29 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses limitations in Supervised Fine-Tuning (SFT) of large language models by introducing the maximum entropy principle and developing a new distribution matching method called GEM. The goal is to create models that capture data effectively while producing more diverse outputs, rather than relying solely on Cross Entropy (CE) loss. SFT with CE loss often leads to overfitting and limited output diversity due to its aggressive updates to the data distribution. To achieve this, the authors develop a novel approach that solves reverse Kullback-Leibler divergence minimization with an entropy regularizer. This method is tested on various benchmarks, demonstrating improved performance and increased output diversity. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps make large language models better at doing specific tasks. Right now, these models use something called Supervised Fine-Tuning to learn new skills. The problem is that this process can cause the model to become too good at one thing and not very good at anything else. To fix this, the authors came up with a new way to make the model work. They call it GEM, which stands for “Generalized Entropy Matching”. This method helps the model capture important information from data while also being more creative in its outputs. The goal is to make these models better at doing lots of different things, rather than just one thing. |
Keywords
» Artificial intelligence » Cross entropy » Fine tuning » Overfitting » Supervised
