Summary of Towards Leveraging Automl For Sustainable Deep Learning: a Multi-objective Hpo Approach on Deep Shift Neural Networks, by Leona Hennig et al.
Towards Leveraging AutoML for Sustainable Deep Learning: A Multi-Objective HPO Approach on Deep Shift Neural Networks
by Leona Hennig, Tanja Tornede, Marius Lindauer
First submitted to arxiv on: 2 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper explores the potential of deep shift neural networks (DSNNs) to reduce computational complexity at inference time, making them a more environmentally friendly and resource-efficient option for Deep Learning (DL) models. By leveraging AutoML techniques and hyperparameter optimization (HPO), the researchers aim to maximize DSNN performance while minimizing resource consumption. The study combines multi-objective (MO) optimization with accuracy and energy consumption as complementary objectives, using state-of-the-art multi-fidelity (MF) HPO methods. Experimental results show that this approach leads to models with over 80% accuracy and low computational cost, accelerating efficient model development and enabling sustainable AI applications. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Deep Learning is a powerful tool that can help us understand the world better. However, making these complex calculations requires a lot of energy and computing power. The researchers are trying to find a way to make Deep Learning more environmentally friendly by using something called deep shift neural networks (DSNNs). They’re also looking for ways to optimize how DSNNs work so they can do their job well but use less energy and resources. To achieve this, they’re combining different techniques that help them find the best settings for their models. The results are very promising – they’ve been able to create models that are over 80% accurate while using much less energy than before. |
Keywords
* Artificial intelligence * Deep learning * Hyperparameter * Inference * Optimization
