Summary of Yoda: Teacher-student Progressive Learning For Language Models, by Jianqiao Lu et al.
YODA: Teacher-Student Progressive Learning for Language Models
by Jianqiao Lu, Wanjun Zhong, Yufei Wang, Zhijiang Guo, Qi Zhu, Wenyong Huang, Yanlin Wang, Fei Mi, Baojun Wang, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu
First submitted to arxiv on: 28 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary YODA, a novel teacher-student progressive learning framework, aims to improve the efficacy of model fine-tuning by emulating the teacher-student education process. The framework operates in an interactive loop, providing tailored feedback on student answers and organizing the education process through basic, generalized, and harder questions. This process is reflected in systematic procedural data used for training models like LLaMA2, which showed significant performance gains (+17.01% on GSM8K and +9.98% on MATH) when trained with YODA data. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary YODA helps machines learn better by mimicking how humans learn from simple examples and get better at solving harder problems. It’s like having a teacher that gives feedback and helps you understand things better. This new way of teaching models can improve their performance by 17-10% on math tests! |
