Summary of Data Value Estimation on Private Gradients, by Zijian Zhou et al.
Data value estimation on private gradients
by Zijian Zhou, Xinyi Xu, Daniela Rus, Bryan Kian Hsiang Low
First submitted to arxiv on: 22 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed method for gradient-based machine learning (ML) methods, such as stochastic gradient descent, perturbs the gradients with random Gaussian noise to enforce differential privacy (DP). Data valuation attributes the ML performance to the training data and is widely used in privacy-aware applications. However, existing data valuation methods cannot be used when DP is enforced via gradient perturbations due to the paradoxical linear scaling of estimation uncertainty with more budget. Instead, injecting carefully correlated noise can provably remove this issue. The proposed method gives better data value estimates on various ML tasks and is applicable to use cases including dataset valuation and federated learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about how to keep machine learning models private when they’re being trained. Right now, a common way to do this is by adding random noise to the “gradients” of the model (which are like its mistakes). But this method doesn’t work well if you want to know how good the training data is. The authors show that this is because the more budget you have to train the model, the worse your estimates become! To fix this, they propose a new way of adding noise that gets rid of this problem. They also test their method on some machine learning tasks and find that it works better. |
Keywords
» Artificial intelligence » Federated learning » Machine learning » Stochastic gradient descent
