Summary of Resource-adaptive Successive Doubling For Hyperparameter Optimization with Large Datasets on High-performance Computing Systems, by Marcel Aach et al.
Resource-Adaptive Successive Doubling for Hyperparameter Optimization with Large Datasets on High-Performance Computing Systems
by Marcel Aach, Rakesh Sarma, Helmut Neukirchen, Morris Riedel, Andreas Lintermann
First submitted to arxiv on: 3 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Distributed, Parallel, and Cluster Computing (cs.DC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Resource-Adaptive Successive Doubling Algorithm (RASDA) combines a resource-adaptive successive doubling scheme with the plain Asynchronous Successive Halving Algorithm (ASHA). This approach is applied to Neural Networks (NNs) and trained on large datasets from various domains, including Computer Vision (CV), Computational Fluid Dynamics (CFD), and Additive Manufacturing (AM). Empirical results show that RASDA outperforms ASHA by a factor of up to 1.9 in terms of runtime while maintaining or surpassing the solution quality of final ASHA models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary RASDA is a new way to speed up the process of finding the best settings for machine learning models on very powerful computers called High-Performance Computing (HPC) systems. Normally, many different combinations of settings are tried and tested, but this takes a long time. RASDA makes it faster by choosing which combinations to test based on how well they’re doing so far. It’s useful because it lets us try more combinations without having to wait too long. This is important for training big neural networks that need lots of data and computing power. |
Keywords
» Artificial intelligence » Machine learning
