Summary of Automated Filtering Of Human Feedback Data For Aligning Text-to-image Diffusion Models, by Yongjin Yang et al.
Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models
by Yongjin Yang, Sihyeon Kim, Hojung Jung, Sangmin Bae, SangMook Kim, Se-Young Yun, Kimin Lee
First submitted to arxiv on: 14 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces FiFA, an automated algorithm for fine-tuning text-to-image diffusion models with human feedback. The main challenge is addressing slow convergence due to noisy feedback datasets. FiFA solves this issue by optimizing three components: preference margin, text quality, and text diversity. Preference margin identifies high-informational value samples using a proxy reward model, while text quality prevents harmful contents through large language models. Text diversity ensures generalization via k-nearest neighbor entropy estimation. The algorithm assigns importance scores to data pairs and selects the most important ones for filtering. Experimental results show that FiFA improves training stability and performance, with human preference increasing by 17%, using only 0.5% of the full dataset and 1% of GPU hours compared to using the full feedback dataset. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you’re trying to teach a computer to create images based on text descriptions. However, this process can be slow because humans provide feedback that’s sometimes noisy or low-quality. This paper proposes an innovative solution called FiFA to speed up this process and make it more effective. FiFA uses clever math to pick the most important parts of the human feedback and ignore the rest. This way, the computer learns faster and makes better images based on what humans want. The results show that FiFA is significantly better than current methods and helps computers create more accurate images with less effort. |
Keywords
» Artificial intelligence » Fine tuning » Generalization » Nearest neighbor
