Summary of Enhancing Neural Network Robustness Against Fault Injection Through Non-linear Weight Transformations, by Ninnart Fuengfusin et al.
Enhancing Neural Network Robustness Against Fault Injection Through Non-linear Weight Transformations
by Ninnart Fuengfusin, Hakaru Tamukoh
First submitted to arxiv on: 28 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to enhancing the robustness of deep neural networks (DNNs) against faults that can occur in physical hardware due to radiation, aging, and temperature fluctuations. Instead of restricting activation ranges using clipped ReLU, this work focuses on constraining DNN weights by applying saturated activation functions (SAFs). SAFs prevent excessive weight growth, which can lead to model failure. The proposed method not only improves robustness against fault injections but also enhances DNN performance with a small margin. To deploy the trained models, the weights are constrained using SAFs and then written to mediums with faults. During inference, the weights with faults are applied with SAFs. The authors demonstrate their approach across three datasets (CIFAR10, CIFAR100, ImageNet 2012) and three data types (32-bit floating point (FP32), 16-bit floating point, and 8-bit fixed point). Notably, they show that their method enables an FP32 ResNet18 with ImageNet 2012 to operate at a bit-error rate of 0.00001 with minor accuracy loss. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making sure artificial intelligence models called deep neural networks (DNNs) work well even when they’re used in real-world devices that can get damaged. Right now, DNNs are sensitive to problems that can happen in the hardware, like radiation or temperature changes. To fix this, some people have tried limiting how much certain parts of the model can change. Instead, this new method focuses on making sure the weights (which are like special numbers) inside the model don’t get too big and cause trouble. This helps both make the models more reliable and also makes them a little bit better at doing their job. The authors tested their idea with different types of data and showed that it can work well even when there’s some “noise” in the system. |
Keywords
» Artificial intelligence » Inference » Relu » Temperature
