Summary of Energy-aware Dynamic Neural Inference, by Marcello Bullo et al.
Energy-Aware Dynamic Neural Inference
by Marcello Bullo, Seifallah Jardak, Pietro Carnelli, Deniz Gündüz
First submitted to arxiv on: 4 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Signal Processing (eess.SP); Systems and Control (eess.SY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an on-device adaptive inference system that can seamlessly integrate deep learning algorithms into energy-limited or energy-harvesting end-devices. This system reduces the run-time execution cost by either switching between differently-sized neural networks (multi-model selection) or enabling earlier predictions at intermediate layers (early exiting). The choice of model and exit point is dynamically made based on the energy storage and harvesting process states. Additionally, the paper studies the integration of prediction confidence into the decision-making process and derives a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers. Experimental results show that energy- and confidence-aware control schemes achieve approximately 5% improvement in accuracy compared to their energy-aware confidence-agnostic counterparts. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how to make devices like smartphones or smart homes use artificial intelligence (AI) without running out of power. The problem is that these devices often don’t have enough energy to do all the AI tasks they need to, so they can’t get the job done. To solve this issue, the researchers propose a system that can adjust how much AI work it does based on how much energy it has left. This helps the device use up less power and stay powered for longer. The paper also shows that if you take into account how sure the AI is about its predictions, it can make even better decisions. |
Keywords
* Artificial intelligence * Deep learning * Inference
