Summary of Variance-reduced Zeroth-order Methods For Fine-tuning Language Models, by Tanmay Gautam et al.
Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by Tanmay Gautam, Youngsuk Park, Hao Zhou, Parameswaran Raman, Wooseok Ha
First submitted to arxiv on: 11 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Fine-tuning language models has shown success in various downstream tasks, but as models scale up, memory requirements become prohibitively high for backpropagation. Zeroth-order (ZO) optimization methods leverage memory-efficient forward passes to estimate gradients. MeZO, an adaptation of ZO-SGD, outperforms zero-shot and in-context learning when combined with suitable task prompts. This work couples ZO methods with variance reduction techniques to enhance stability and convergence for inference-based LM fine-tuning. Introducing Memory-Efficient Zeroth-Order Stochastic Variance-Reduced Gradient (MeZO-SVRG), the paper demonstrates its efficacy across multiple LM fine-tuning tasks, eliminating reliance on task-specific prompts. MeZO-SVRG outperforms MeZO with up to 20% increase in test accuracies on benchmark GLUE tasks, reducing computation time by 2and memory footprint compared to first-order SGD. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper talks about how we can make language models better. Right now, making these models work is hard because they need too much computer memory. The researchers found a way to fix this problem using something called zeroth-order optimization methods. They combined these methods with other techniques to make the process more stable and efficient. This new method is called MeZO-SVRG, and it can be used for many different language model fine-tuning tasks without needing special prompts. It even works faster and uses less memory than previous methods! |
Keywords
» Artificial intelligence » Backpropagation » Fine tuning » Inference » Language model » Optimization » Zero shot
