Summary of On Gradient Descent Ascent For Nonconvex-concave Minimax Problems, by Tianyi Lin et al.
On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
by Tianyi Lin, Chi Jin, Michael I. Jordan
First submitted to arxiv on: 2 Jun 2019
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Optimization and Control (math.OC); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles a fundamental challenge in machine learning: solving nonconvex-concave minimax problems. These problems involve minimizing a non-convex function while maximizing a concave one, which is crucial for training generative adversarial networks (GANs) and other applications. The gradient descent ascent (GDA) algorithm is widely used to solve this problem, but its convergence properties are not well understood in general settings. In fact, GDA with equal stepsize can converge to limit cycles or even diverge. To address this issue, the authors propose a two-time-scale version of GDA and provide non-asymptotic complexity results for solving nonconvex-concave minimax problems efficiently. This breakthrough analysis sheds light on the superior practical performance of two-time-scale GDA in training GANs and other real applications. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a type of math problem that’s important for machine learning. It’s called a “minimax” problem, which means we’re trying to find the best solution by minimizing one thing while maximizing another. The problem gets tricky when the functions involved are not easy to work with. The authors propose a new way to solve this problem using an algorithm called GDA, and they show that it can be very effective in certain situations. This is important because we need efficient ways to train machines that can learn from data. The paper’s results will help us understand how to use GDA better, which could lead to breakthroughs in fields like image recognition. |
Keywords
* Artificial intelligence * Gradient descent * Machine learning
