Summary of Xmlp: Revolutionizing Private Inference with Exclusive Square Activation, by Jiajie Li et al.
xMLP: Revolutionizing Private Inference with Exclusive Square Activation
by Jiajie Li, Jinjun Xiong
First submitted to arxiv on: 12 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Cryptography and Security (cs.CR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Private Inference (PI) enables deep neural networks (DNNs) to operate on private data without compromising sensitive information. The use of non-linear activations, such as ReLU, can result in impractically high PI latency due to the requirement for costly multi-party computation (MPC) or homomorphic encryption (HE). This paper explores the accuracy drop when using square activations, attributing it to an “information compounding” effect. The authors propose xMLP, a novel DNN architecture that exclusively uses square activations while maintaining parity in both accuracy and efficiency with ReLU-based models. Experiments on CIFAR-100 and ImageNet demonstrate superior performance of xMLP models compared to ResNet variants, achieving better performance with fewer activation layers and parameters. When offloading PI to the GPU, xMLP is up to 700x faster than previous state-of-the-art PI models while maintaining comparable accuracy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine being able to use powerful artificial intelligence (AI) without putting sensitive information at risk. This paper shows how AI can work on private data without leaking secrets by using special math techniques called “cryptography”. The problem is that these techniques make the AI very slow, so scientists have developed a new way of making AI work faster while keeping it accurate. This breakthrough could revolutionize many areas, from healthcare to finance, where secrecy is crucial. |
Keywords
* Artificial intelligence * Inference * Relu * Resnet
