Summary of What to Do When Your Discrete Optimization Is the Size Of a Neural Network?, by Hugo Silva and Martha White
What to Do When Your Discrete Optimization Is the Size of a Neural Network?
by Hugo Silva, Martha White
First submitted to arxiv on: 15 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Machine learning applications often rely on discrete optimization techniques to solve complex problems. However, these approaches can be inefficient when dealing with large neural networks. To address this challenge, researchers have developed two main classes of methods: continuation path (CP) and Monte Carlo (MC). CP methods use gradient information from outside the solution set, while MC methods compare evaluations between valid solutions. This paper compares the performance of both approaches using smaller microworld experiments and larger problems like neural network regression and image classification with pruning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Machine learning is used to help computers learn and make decisions on their own. Sometimes, this process involves solving complex math problems. The problem is that these math problems can be really hard for computers to solve when dealing with large amounts of data. To fix this issue, researchers have developed two main ways to solve these problems: CP and MC methods. CP methods use information from outside the solution set, while MC methods compare different solutions. This paper compares how well these methods work using smaller experiments and larger real-world problems. |
Keywords
* Artificial intelligence * Image classification * Machine learning * Neural network * Optimization * Pruning * Regression
