Summary of Low-energy On-device Personalization For Mcus, by Yushan Huang et al.
Low-Energy On-Device Personalization for MCUs
by Yushan Huang, Ranya Aloufi, Xavier Cadet, Yuchen Zhao, Payam Barnaghi, Hamed Haddadi
First submitted to arxiv on: 12 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Hardware Architecture (cs.AR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper introduces MicroT, an efficient and low-energy MCU personalization approach that enables customized models to enhance task adaptation without requiring complex local pre-training or training. This approach includes a tiny feature extractor developed through self-supervised knowledge distillation, which trains a task-specific head for on-device personalization with minimal energy and computational requirements. The paper also implements an MCU-optimized early-exit inference mechanism called stage-decision to reduce energy costs. Evaluations using two models, three datasets, and two MCU boards show that MicroT outperforms traditional transfer learning (TTL) and two SOTA approaches by 2.12 – 11.60% across two models and three datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper introduces a new approach for personalizing machine learning models on microcontrollers called MicroT. This approach makes it possible to train customized models on devices that don’t have much power or storage space. The model uses a special kind of training called self-supervised knowledge distillation, which helps it learn quickly and efficiently. The authors also developed a way to stop the model from using too much energy during inference, by allowing it to “exit” early if it’s already close to making an accurate prediction. They tested MicroT on two different models, three datasets, and two microcontrollers, and found that it outperformed other approaches. |
Keywords
* Artificial intelligence * Inference * Knowledge distillation * Machine learning * Self supervised * Transfer learning
