Summary of Learning Privacy-preserving Student Networks Via Discriminative-generative Distillation, by Shiming Ge et al.
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation
by Shiming Ge, Bochao Liu, Pengju Wang, Yong Li, Dan Zeng
First submitted to arxiv on: 4 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to learn privacy-preserving deep models that effectively balance utility and privacy. The authors introduce a discriminative-generative distillation method, which leverages multiple teachers trained on private data and a generator trained in a data-free manner. This approach generates synthetic data, which are then used to train a variational autoencoder (VAE) and query labels via differentially private aggregation. A semi-supervised student learning framework is employed to transfer knowledge from the teachers while enhancing accuracy using tangent-normal adversarial regularization. The authors demonstrate the effectiveness of their approach through extensive experiments and analysis. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes an innovative way to create privacy-preserving deep models that balance utility and privacy. It uses a method called discriminative-generative distillation, which takes data from private sources and generates new synthetic data that can be used without risking personal information. The authors show how their approach can control the amount of private data used and still get good results. |
Keywords
» Artificial intelligence » Distillation » Regularization » Semi supervised » Synthetic data » Variational autoencoder
